DETAILED ACTION
This communication is a Final Rejection Office Action in response to the 4/9/2026 filling of Application 18/928,233. Claims 1-20 are now presented.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 4/9/2026 have been fully considered but they are not persuasive.
Regarding the rejection under 101, the Applicant argues “The Applicant respectfully submits that one or more features of amended independent claim 1 cannot be performed in the human mind and are inherently tied to a specific technological implementation involving advanced data processing and machine learning techniques. The amended claim explicitly recites transformation of textual recall data and risk indicator dictionaries into numerical representations using an machine learning model, followed by determination of risk scores based on a degree of similarity between semantically expanded risk indicators and textual recall reasons. Such operations require specialized computational techniques, including high- dimensional vector representations and similarity computations across large datasets, which cannot be practically performed mentally or using pen and paper. Further, the claimed system performs semantic normalization of heterogeneous textual recall data through generation of keyword lists comprising semantically similar terms for each risk indicator and organizes these into structured dictionaries. This enables the system to identify and correlate linguistically varied expressions across large-scale recall datasets. The process of generating semantically related keyword mappings, transforming them into numerical embeddings, and computing similarity scores across multi-dimensional vector spaces constitutes a technical data processing pipeline that is far beyond mere mental evaluation or generic data analysis. Moreover, the amended claim recites a specific training mechanism for the machine learning model, wherein structured training data is generated based on computed similarity-driven risk scores. This is not a result-oriented recitation but instead defines a concrete technical process for preparing and utilizing training data, including transformation, mapping, and scoring operations. The claimed approach improves the functioning of computer systems by enabling accurate processing of unstructured textual recall data through semantic alignment and numerical representation, thereby enhancing prediction accuracy for recall risks.”
The Examiner respectfully disagrees. The generating a list of keywords for each of the general risk indicators and specific risk indicators; creating a general and specific risk indicator dictionary comprising a grouping of each of the general risk indicators with their corresponding list of keywords; and providing the recall data, general risk indicator dictionary and specific risk indicator dictionary to a model for predicting risk of recall of a product are directed to mental processes. The Applicant argues that “transformation of textual recall data and risk indicator dictionaries into numerical representations using an machine learning model, followed by determination of risk scores based on a degree of similarity between semantically expanded risk indicators and textual recall reasons” cannot be performed mentally. The technical requirements argued by the Applicant (including high- dimensional vector representations and similarity computations across large datasets) are not recited in the claims. Under the broadest reasonable interpretation the transformation of textual recall data and risk indicator dictionaries into numerical representations can be performed mentally. Further, the machine learning s recited broadly and amounts to attempt to cover any solution to the identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result.
Regarding the rejection under 101, the Applicant argues “the claimed system retrieves textual reasons for recall from one or more recall databases and, through the generative module, creates keyword lists comprising semantically similar terms associated with each risk indicator. The training module performs a transformation of the recall data and the dictionary data into numerical representations using the machine learning model, and further determines risk scores based on a degree of similarity between the numerical representations of risk indicators and textual recall reasons. This process constitutes a concrete data transformation and structured training data generation mechanism that enables automated semantic alignment between disparate textual inputs and predefined risk criteria. For example, in a product safety analysis environment, recall reports from different regulatory sources may describe similar issues using varied terminology. The claimed system enables these variations to be captured through semantically generated keyword mappings and encoded into numerical representations, allowing the machine learning model to consistently identify and quantify risk patterns across diverse textual inputs. The computed similarity-based risk scores are then used as structured training data, enabling accurate and scalable prediction of recall risks for new products. Accordingly, the amended claim provides a specific technological improvement in the processing of unstructured textual data and machine learning model training, as it enables structured storage, transformation, and automated utilization of semantically enriched risk indicator mappings for recall risk prediction.”
The Examiner respectfully disagrees. Limitation that can be classified into abstract idea groupings does cannot also be directed to technical improvements. Viewing the additional elements of the generic data gathering and broadly recited machine learning model training and model implementation in combination with the generic computer does not add more than when viewing the elements individually. Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding the rejection under 101, the Applicant argues “the subject matter of amended independent claim 1 provides significant technical advantages in view of the limitations of conventional recall risk assessment systems. The claimed system introduces a technologically advanced framework for processing large-scale, unstructured textual recall data by generating semantically enriched keyword mappings for user- defined general and specific risk indicators and organizing them into structured dictionaries. These dictionaries enable consistent interpretation of linguistically varied recall descriptions across multiple data sources. The system further performs automated transformation of textual recall data and dictionary data into numerical representations using a machine learning model, followed by computation of similarity-based risk scores that quantify the relationship between risk indicators and textual recall reasons. This structured transformation and similarity-driven scoring mechanism enables efficient preparation of training data and improves the accuracy and scalability of recall risk prediction. As a result, the claimed system enhances the functioning of computer-based data processing by enabling reliable semantic alignment, automated analysis of heterogeneous textual datasets, and improved predictive modeling for product recall risks, thereby reducing reliance on manual analysis and improving decision-making efficiency in product safety management.”
The Examiner respectfully disagrees. Limitation that can be classified into abstract idea groupings does cannot also be directed to technical improvements. Viewing the additional elements of the generic data gathering and broadly recited machine learning model training and model implementation in combination with the generic computer does not add more than when viewing the elements individually. Accordingly, the additional elements do not provide and inventive concept.
Regarding the rejection under 103, the Applicant argues “Leidner describe about mining and structuring risk information from unstructured text sources to produce risk profiles and related predictive insights. Nowhere does Leidner teaches or suggest "one or more textual reasons for recall" as claimed. Instead, Leidner processes general news/media content to identify risk-related statements. The risks in Leidner are extracted from broad textual sources and are not tied to product recall contexts or recall-specific reasoning. Further, while Leidner discloses identification of risk-related terms and taxonomies, it does not teach generation of semantically similar keywords for each user-defined risk indicator as claimed. Whereas the claimed subject matter specifically generates keyword lists corresponding to risk indicators to account for linguistic variations, whereas Leidner merely extracts terms of present in the corpus and relies on statistical associations or predefined lexicons. There is no teaching in Leidner of creating a mapping between a risk indicator and a plurality of semantically similar terms for downstream training purposes. Most importantly, Nowhere does Leidner describe transforming textual recall data into numerical representations and determining risk scores based on a degree of similarity as claimed in the amended independent claim 1. Instead, Leidner relies on frequency counts, polarity scoring, and pattern matching for risk identification. At last, Leidner's machine learning model is trained on financial data and media signals, and not on a structured dataset derived from semantic similarity between recall reasons and risk indicators, as recited in the amended independent claim 1.”
The Examiner respectfully disagrees. Leidner teaches analyzing textual reasons for entity risk. Sen teaches using machine learning to predict medical device product risks. Both Leidner and Sen are directed to parsing unstructured data to identify risk. As the Examiner explained in the rejection, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis by including accessing at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories and the risk is associated with a product recall and the categories are product categories as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls. As such, the combination of Leidner and Sen teach :generate a list of keywords for each of the general risk indicators and specific risk indicators, a keyword comprising one or more terms semantically similar to the corresponding risk indicator to account for linguistic variations in the textual reasons for recall and transforming textual recall data into numerical representations and determining risk scores based on a degree of similarity.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
When considering subject matter eligibility under 35 U.S.C. 101, in step 1 it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. If the claim does fall within one of the statutory categories, in step 2A prong 1 it must then be determined whether the claim is recite a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea). If the claim recites a judicial exception, under step 2A prong 2 it must additionally be determined whether the recites additional elements that integrate the judicial exception into a practical application. If a claim does not integrate the Abstract idea into a practical application, under step 2B it must then be determined if the claim provides an inventive concept.
In the Instant case, Claims 1-8 are directed toward a system for predicting risk of recall of a product. Claims 9-14, 18-20 are directed toward a method for predicting risk of recall of a product. Claims 15-17 are directed toward a computer program product for predicting risk of recall of a product. As such, each of the Claims is directed to one of the four statutory categories of invention.
MPEP 2106.04 II. A. explains that in step 2A prong 1 Examiners are to determine whether a claim recites a judicial exception. MPEP 2106.04(a) explains that:
To facilitate examination, the Office has set forth an approach to identifying abstract ideas that distills the relevant case law into enumerated groupings of abstract ideas. The enumerated groupings are firmly rooted in Supreme Court precedent as well as Federal Circuit decisions interpreting that precedent, as is explained in MPEP § 2106.04(a)(2). This approach represents a shift from the former case-comparison approach that required examiners to rely on individual judicial cases when determining whether a claim recites an abstract idea. By grouping the abstract ideas, the examiners’ focus has been shifted from relying on individual cases to generally applying the wide body of case law spanning all technologies and claim types.
The enumerated groupings of abstract ideas are defined as:
1) Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP § 2106.04(a)(2), subsection I);
2) Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (see MPEP § 2106.04(a)(2), subsection II); and
3) Mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III).
As per step 2A prong 1 of the eligibility analysis, claim 1 recites the abstract idea of predict product recall risk which falls into the abstract idea categories of certain methods of organizing human activity and mental processes. The elements of Claim 1 that represent the Abstract idea include:
A system for predicting risk of recall of a product, comprising:
receive from a user:
general risk indicators corresponding to the plurality of product categories, wherein a general risk indicator is indicative of a user-defined criteria for assessing risk of recall of the plurality of product categories; and
specific risk indicators corresponding to each product category from amongst the plurality of product categories, wherein a specific risk indicator is indicative of a user-defined criteria for assessing risk of recall of the corresponding product category;
a generative module coupled to one or more processors to:
generate a list of keywords for each of the general risk indicators and specific risk indicators, a keyword comprising one or more terms semantically similar to the corresponding risk indicator to account for linguistic variations in the textual reasons for recall;
create a general risk indicator dictionary comprising a grouping of each of the general risk indicators with their corresponding list of keywords; and
create a specific risk indicator dictionary comprising a grouping of each of the specific risk indicators with their corresponding list of keywords; and
provide the recall data, general risk indicator dictionary and specific risk indicator dictionary to a model for predicting risk of recall of a product
transform the recall data and the general risk indicator dictionary and the specific risk indicator dictionary into numerical representation using the model; and
determine, a risk score for each of the general risk indicators and specific risk indicators based on a degree of similarity between the corresponding risk indicator and the one or more textual reasons for recall.
MPEP 2106.04(a)(2) states:
The courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 (2012) ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions
The instant claims recite mental processes including observation, evaluation, judgment, opinion. For example, the generating a list of keywords for each of the general risk indicators and specific risk indicators; creating a general and specific risk indicator dictionary comprising a grouping of each of the general risk indicators with their corresponding list of keywords; and providing the recall data, general risk indicator dictionary and specific risk indicator dictionary to a model for predicting risk of recall of a product are directed to mental processes. There is nothing in the claims that precludes these steps from being performed mentally. As such, the claims recite abstract ideas.
MPEP 2106.04(a)(2) II. states:
The phrase "methods of organizing human activity" is used to describe concepts relating to:
fundamental economic principles or practices (including hedging, insurance, mitigating risk);
commercial or legal interactions (including agreements in the form of contracts, legal obligations, advertising, marketing or sales activities or behaviors, and business relations); and
managing personal behavior or relationships or interactions between people, (including social activities, teaching, and following rules or instructions).
The Supreme Court has identified a number of concepts falling within the "certain methods of organizing human activity" grouping as abstract ideas. In particular, in Alice, the Court concluded that the use of a third party to mediate settlement risk is a ‘‘fundamental economic practice’’ and thus an abstract idea. 573 U.S. at 219–20, 110 USPQ2d at 1982. In addition, the Court in Alice described the concept of risk hedging identified as an abstract idea in Bilski as ‘‘a method of organizing human activity’’. Id. Previously, in Bilski, the Court concluded that hedging is a ‘‘fundamental economic practice’’ and therefore an abstract idea. 561 U.S. at 611–612, 95 USPQ2d at 1010.
In the instant case, the limitations of receive from a user: general risk indicators corresponding to the plurality of product categories, wherein a general risk indicator is indicative of a user-defined criteria for assessing risk of recall of the plurality of product categories; and specific risk indicators corresponding to each product category from amongst the plurality of product categories, wherein a specific risk indicator is indicative of a user-defined criteria for assessing risk of recall of the corresponding product category to product recall risk are directed to commercial interactions including marketing or sales activities or behaviors, and business relations which are abstract.
Under step 2A prong 2 the examiner must then determine if the recited abstract idea is integrated into a practical application. MPEP 2106.04 states:
Limitations the courts have found indicative that an additional element (or combination of elements) may have integrated the exception into a practical application include:
• An improvement in the functioning of a computer, or an improvement to other technology or technical field, as discussed in MPEP §§ 2106.04(d)(1) and 2106.05(a);
• Applying or using a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, as discussed in MPEP § 2106.04(d)(2);
• Implementing a judicial exception with, or using a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim, as discussed in MPEP § 2106.05(b);
• Effecting a transformation or reduction of a particular article to a different state or thing, as discussed in MPEP § 2106.05(c); and
• Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception, as discussed in MPEP § 2106.05(e)
The courts have also identified limitations that did not integrate a judicial exception into a practical application:
• Merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
• Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); and
• Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).
In the instant case, this judicial exception is not integrated into a practical application. In particular, Claim 1 recites the additional elements of:
A system for predicting risk of recall of a product, comprising:
one or more processors to perform the abstract idea;
a data retrieval module coupled to the one or more processors, wherein the data retrieval module is to access at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories;
a training module coupled to the one or more processors and a machine learning (ML) model as training data, to train the ML model
using the ML model to transform the data
However, the processor is recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Further MPEP 2105.05(g) explains that data gathering and data output can be considered pre-solution activity and post-solution activity. See MPEP 2106.05(g) that states:
An example of pre-solution activity is a step of gathering data for use in a claimed process, e.g., a step of obtaining information about credit card transactions, which is recited as part of a claimed process of analyzing and manipulating the gathered information by a series of steps in order to detect whether the transactions were fraudulent. An example of post-solution activity is an element that is not integrated into the claim as a whole, e.g., a printer that is used to output a report of fraudulent transactions, which is recited in a claim to a computer programmed to analyze and manipulate information about credit card transactions in order to detect whether the transactions were fraudulent.
In the instant case, the claims do not provide any particular way that the data is retrieved. As such, the broadly recited retrieval of data amounts to insignificant pre-solution activity.
Further, training and use of the ML models indicative of adding the words “apply it” (or an equivalent) with the judicial exception. MPEP 2106.05(f) states:
When determining whether a claim simply recites a judicial exception with the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer, examiners may consider the following:
(1) Whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished. The recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it". See Electric Power Group, LLC v. Alstom, S.A., 830 F.3d 1350, 1356, 119 USPQ2d 1739, 1743-44 (Fed. Cir. 2016); Intellectual Ventures I v. Symantec, 838 F.3d 1307, 1327, 120 USPQ2d 1353, 1366 (Fed. Cir. 2016); Internet Patents Corp. v. Active Network, Inc., 790 F.3d 1343, 1348, 115 USPQ2d 1414, 1417 (Fed. Cir. 2015). In contrast, claiming a particular solution to a problem or a particular way to achieve a desired outcome may integrate the judicial exception into a practical application or provide significantly more. See Electric Power, 830 F.3d at 1356, 119 USPQ2d at 1743.
By way of example, in Intellectual Ventures I v. Capital One Fin. Corp., 850 F.3d 1332, 121 USPQ2d 1940 (Fed. Cir. 2017), the steps in the claims described "the creation of a dynamic document based upon ‘management record types’ and ‘primary record types.’" 850 F.3d at 1339-40; 121 USPQ2d at 1945-46. The claims were found to be directed to the abstract idea of "collecting, displaying, and manipulating data." 850 F.3d at 1340; 121 USPQ2d at 1946. In addition to the abstract idea, the claims also recited the additional element of modifying the underlying XML document in response to modifications made in the dynamic document. 850 F.3d at 1342; 121 USPQ2d at 1947-48. Although the claims purported to modify the underlying XML document in response to modifications made in the dynamic document, nothing in the claims indicated what specific steps were undertaken other than merely using the abstract idea in the context of XML documents. The court thus held the claims ineligible, because the additional limitations provided only a result-oriented solution and lacked details as to how the computer performed the modifications, which was equivalent to the words "apply it". 850 F.3d at 1341-42; 121 USPQ2d at 1947-48 (citing Electric Power Group., 830 F.3d at 1356, 1356, USPQ2d at 1743-44 (cautioning against claims "so result focused, so functional, as to effectively cover any solution to an identified problem")).
In the instant case, the additional elements of the broadly recited training and use of the ML model attempt to cover any solution to the identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, which does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it”. For example, the claims do not state how the model is trained; how it is used to predict recall risk or to transform the data into a numerical representation. As such, the broadly recited ML model does not integrate a judicial exception into a practical application or provide significantly more.
Viewing the generic data gathering and broadly recited machine learning model training and model implementation in combination with the generic computer does not add more than when viewing the elements individually. Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
In step 2B, the examiner must determine whether the claim adds a specific limitation other than what is well-understood, routine, conventional activity in the field - see MPEP 2106.05(d). As discussed with respect to Step 2A Prong Two, the processing circuitry in the claim amount to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. Accordingly, the additional elements do not provide and inventive concept.
Further, nothing in the specification indicates that the retrieving of data is anything other than conventional. Further, MPEP 2106.05(d) states “Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."” Further, MPEP 2106.05(d) also states that creating output data has been identified as conventional (see Return Mail, Inc. v. U.S. Postal Service, -- F.3d --, -- USPQ2d --, slip op. at 32 (Fed. Cir. August 28, 2017)).
Further, similar to the analysis with respect to step 2A prong 2 recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished cannot provide an inventive concept under step 2B of the eligibility analysis.
Viewing the generic data gathering and broadly recited model training in combination with the generic computer does not add more than when viewing the elements individually. Accordingly, the additional elements do provide and inventive concept.
Further, Claims 2-8 further limit the mental processes and methods of organizing human activity already rejected in the parent claim, but fail to remedy the deficiencies of the parent claim as they do not impose any additional elements that amount to significantly more than the abstract idea itself. Further, claims 2-4 further define the outputs of the ML model, but do not recite how those outputs are obtained.
Further, claims 6 and 8 recited the use of a LLM. However, similar to the ML model, the use of the LLM amounts to adding the words “apply it” (or an equivalent) with the judicial exception.
Accordingly, the Examiner concludes that there are no meaningful limitations in claims 1-8 that transform the judicial exception into a patent eligible application such that the claim amounts to significantly more than the judicial exception itself.
The analysis above applies to all statutory categories of invention. The presentment of claim 1 otherwise styled as a computer program product, or method for example, would be subject to the same analysis. As such, claims 9-17 are also rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 3, 5, 9, 15, 10, 16, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leidner US 2012/0221485 A1 in view of Sen US 2024/0233932 A1.
As per Claim 1 Leidner teaches a system for predicting risk of recall of a product, comprising:
one or more processors;
a data retrieval module coupled to the one or more processors, wherein the data retrieval module is to access at least one database to obtain data pertaining to a plurality of categories, the data comprising one or more reasons relating to each of the plurality of categories; such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices. Leidner para. 75 teaches the ERPGS may further process information, including historical trading information, historical risk information, and historical ERP and risk scores to arrive at an anticipated or predictive behavior of stock price and other investment vehicles. The ERPGS 1000 leverages traditional and new media resources to provide a risk-based solution that expands the scope of conventional tools to provide an enhanced analysis data structure for use by financial analysts, investment managers, risk managers and others. Para. 77 teaches The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
an input module coupled to one or more processors, wherein the input module is to receive from a user:
Leidner para. 45 teaches These generated patterns can be used for re-processing the corpus, in one embodiment of the present invention after some human review, or automatically in another embodiment. The extracted sentences or sentence sequences are then both validated (whether or not they are really risk-indicating sentences) and parsed into risks of the form P=>Q (i.e. finding out which text spans correspond to the precondition "P", which parts express the implication "=>", and which parts express the high-impact event "Q"), using, but not limited to, the following non-limiting features: a set of terms with significant statistical association with the term "risk" (in one embodiment of this invention, statistical programs, such as Pointwise Mutual Information (PMI) and Log Likelihood, or rules, including but not limited to rules obtained by Hearst pattern induction, may be used to determine the set of terms); a set of binary gazetteer features, where the feature fires if a gazetteer a set of risk-indicative terms ("threat", "bankruptcy", "risk", . . . ) compiled by human experts or extracted from hand-labeled training data; a set of indicators of speculative language; instances of future time reference; occurrences of conditionals; and/or occurrences of causality markers.
general risk indicators corresponding to the plurality of categories, wherein a general risk indicator is indicative of a user-defined criteria for assessing risk of the plurality categories; and
Leidner para. 90 teaches In one exemplary method of the present invention, and with reference to FIG. 17A, a method for generating an ERP 3000 is illustrated as follows. Initially, at step 3010, the ERP system obtains information and content of interest from credible news/media sources (news feeds, blogs, websites, etc.) from internal or external sources. At step 3020 the ERP system applies linguistic computational analysis and learned risk taxonomy to the obtained information from step 3010 to identify risks and entities referenced or mentioned in the content. At step 3030, the ERP system associates risks with one or more entities identified in step 3020. At step 3040, the system applies a risk taxonomy to arrive at a separate score or indication or a derivative score or indication for at least two risk components: General or generic risks; idiosyncratic risks; self-trend; and peer trend. [0119] Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high.
specific risk indicators corresponding to each category from amongst the plurality of categories, wherein a specific risk indicator is indicative of a user-defined criteria for assessing risk of the corresponding category; Leidner para. 120 teaches FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system.
a generative module coupled to one or more processors to:
generate a list of keywords for each of the general risk indicators and specific risk indicators, a keyword comprising one or more terms semantically similar to the corresponding risk indicator to account for linguistic variations in the textual reasons for risk; Leidner para. 119-120 teaches Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high. FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system. Further, para. 45 teaches in one embodiment of the invention, trigger keywords are used (e.g. "risk", "threat") to generate the risk database. In another embodiment, regular expressions are used (e.g. "("may")? pose(s)? (a)? threat(s)? to") to generate the risk database. Candidate risk sentences or sentence sequences are created, and new patterns are generalized by running a named entity tagger or Part of Speech (POS) tagger, and chunker (entities can be described by proper nouns or NNPs, and not just given by named entities) over it, and by substituting entities by per-class placeholder (e.g. "J.P. Morgan"=>"<COMPANY>"). These generated patterns can be used for re-processing the corpus, in one embodiment of the present invention after some human review, or automatically in another embodiment. The extracted sentences or sentence sequences are then both validated (whether or not they are really risk-indicating sentences) and parsed into risks of the form P=>Q (i.e. finding out which text spans correspond to the precondition "P", which parts express the implication "=>", and which parts express the high-impact event "Q"), using, but not limited to, the following non-limiting features: a set of terms with significant statistical association with the term "risk" (in one embodiment of this invention, statistical programs, such as Pointwise Mutual Information (PMI) and Log Likelihood, or rules, including but not limited to rules obtained by Hearst pattern induction, may be used to determine the set of terms); a set of binary gazetteer features, where the feature fires if a gazetteer a set of risk-indicative terms ("threat", "bankruptcy", "risk", . . . ) compiled by human experts or extracted from hand-labeled training data; a set of indicators of speculative language; instances of future time reference; occurrences of conditionals; and/or occurrences of causality markers.
create a general risk indicator dictionary comprising a grouping of each of the general risk indicators with their corresponding list of keywords; and Leidner para. 119-120 teaches now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high. [0106] In implementation, risk mining may include applying Web mining and information extraction to learning a taxonomy of risk types with little supervision. As discussed above, linguistic patterns are deployed with modifications to determine risk types in an iterative way, e.g., risk such as financial risk type. The data may then be "stuffed" back into an original query pattern. For example, additional more specific terms, e.g., "financial risk," may be arrived at by building from more general terms, e.g., "risk." One manner of achieving this building of terms is by use of an iterative approach using Hearst pattern induction. The system learns to take action upon encountering these terms in a new document.
create a specific risk indicator dictionary comprising a grouping of each of the specific risk indicators with their corresponding list of keywords; and Leidner para. 119-120 teaches now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high. FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system. [0122] In addition, content appearing in a document from the corpus may be identified with multiple entities and may be identified as risks with multiple entities. For instance, an idiosyncratic risk "labor disruption" may be included in a list of such risks, e.g., list 2202 of FIG. 20.
a training module coupled to the one or more processors, to provide the data, general risk indicator dictionary and specific risk indicator dictionary to a machine learning (ML) model as training data, to train the ML model for predicting risk, wherein the training the ML model comprises Leidner para. 77 teaches The ERPGS 1000 of FIG. 15 includes risk scoring and ERP generating module 1250 adapted to process news/media information received as input via news/media corpus 1100 and to identify risks associated with particular entities and arrive at risk scoring in processing news/media items related to one or more companies. ERP and risk score may be derived from computational linguistics and define or represent credible statements identified from, e.g., an article. The risk, as discussed in more detail below, will be interpreted as either positive, negative or neutral, and assigned respective polarizations, e.g., scores of +1, -1, and 0. The score may be derived from text and/or metadata from news/media and may apply a predefined or learned lexicon-based risk taxonomy or pattern to the processed text/metadata. The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
transform the recall data and the general risk indicator dictionary and the specific risk indicator dictionary into numerical representation using the ML model; and Leidner para. 123-124 teach with respect to FIGS. 21 and 22, FIG. 21 is a graphical representation of a Self Trend. In this exemplary embodiment, "Self Trend" represents a time series including the sum of General and Idiosyncratic risk counts or other scoring. In this example the risk counts or scoring are normalized and with a moving average applied to them. For Self Trend, the ERP system considers historical data, in this example data over the preceding 200 days, and normalizes the data against an entity's own past. In this example, the graph represents opportunities/positive risks 2302 and threats/negative risks 2304 identified and quantified from a content collection as being associated with an entity.
FIG. 22 is a graphical representation of a Peer Trend. In this exemplary embodiment, Peer Trend represents a time series including the sum of absolute values of General and Idiosyncratic risk counts or other scoring, i.e., threats and opportunities are dealt with separately. In this example the risk counts or scoring are normalized and with a moving average applied to them. For Peer Trend, the ERP system considers historical data, in this example data over the preceding 200 days, and normalizes the data against the average of the entity's industry or sector peers. In this example, the graph represents opportunities/positive risks 2402 and threats/negative risks 2404 identified and quantified from a content collection as being associated with a peer group. The peer group may or may not include the subject entity. Also, a financial analyst may weigh relative risk exposure associated with a peer group more heavily than a company's self trend in isolation. In any event, the analyst can review this as a separate risk aspect of the entity. This is another example of the robustness and versatility of the multi-component ERP profile of the present invention and its various beneficial uses.
determine, during training, a risk score for each of the general risk indicators and specific risk indicators based on a degree of similarity between the corresponding risk indicator and the one or more textual reasons for recall. Leidner para. 77 teaches The ERPGS 1000 of FIG. 15 includes risk scoring and ERP generating module 1250 adapted to process news/media information received as input via news/media corpus 1100 and to identify risks associated with particular entities and arrive at risk scoring in processing news/media items related to one or more companies. ERP and risk score may be derived from computational linguistics and define or represent credible statements identified from, e.g., an article. The risk, as discussed in more detail below, will be interpreted as either positive, negative or neutral, and assigned respective polarizations, e.g., scores of +1, -1, and 0. The score may be derived from text and/or metadata from news/media and may apply a predefined or learned lexicon-based risk taxonomy or pattern to the processed text/metadata. The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events. Pars. 46 teaches In one embodiment of the present invention, a variant of surrogate machine-learning (i.e., technology for machine learning tasks by examples) may be used to create training data for a machine-learning based classifier that extracts risk-indicative sentences
Leidner does not teach access at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories However, Sen para. 22 teaches the systems and methods described in the present disclosure provide several advantages over existing systems for monitoring medical device recalls. As one example, the disclosed systems and methods provide analytical insights within predicate device and device recall databases that are created from data extracted from regulatory authority data sources. For instance, the predicted probabilities of medical device recalls across different time windows (e.g., probability of recalls in 2-year or 3-year time windows) can be estimated. This insight is advantageous for medical device manufacturers and regulatory bodies as they consider marketing new medical devices through the 510(k) clearance pathway, or monitoring the chances of recalls for existing medical devices. As another example, the disclosed systems and methods create a predicate network for each medical device (including its predecessors and successors) that not only presents the citing relationship among devices but also shows which devices are recalled or have a high recall probability. The shortest citing path (in terms of network hops and/or approval year gap) from the focal device to the recalled device can also be computed and visualized to help evaluate the recall probability of the focal device. As yet another example, the disclosed systems and methods provide database creation and visualization processes that are automatic without requiring manual data scraping from data sources websites, making the analysis and visualization tasks scalable. Further, para. 31 teaches Data collection and extraction 102 includes steps and components for constructing and/or updating a predicate device database 120, constructing and/or updating a device adverse events database 122, constructing and/or updating a recall database 123, constructing and/or updating one or more device clearance records 124 that may contain additional medical device features retrieved from other data sources. Adverse events database 122 includes adverse events such as deaths and injuries associated with all 510(k) devices including the number of injuries, the number of deaths and the number of malfunctions associated with each device. Recall database 123 contains the recall history (e.g. recall date, resolving date, recall type, current recall status) for each device that has been recalled. Device clearance records 124 include basic device information such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices.
Leidner does not teach the analyzed risk is associated with a product recall and the categories are product categories. However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Further par. 49-51 teach [0049] Feature construction 103 uses a selected device 108, referred to as the focal device, and predicate database 120, device recall database 123, device adverse events database 122 and device clearance records 124 to construct a set of features that can be applied to the predictive model. These features are divided into predicate features that are derived from attributes of predicate devices of a device and stand alone features that are derived only from attributes of the device. Further, some of the features are static features (time-invariant in that they do not change from year to year, while other features are time-varying. Time-varying features are computed by year.
To form the predicate features, a predicate network 130 is formed for selected device 108 using predicate database 120. In predicate network 130, devices that are connected directly to selected device 108 are said to be one hop from selected device 108, devices that are connected to selected device 108 through one intermediary device are said to be two hops from selected device 108 and devices that are connected to device 108 through two intermediary devices are said to be three hops from selected device 108. If there are two paths between a device and selected device 108, the shortest path is used.
Predicate features and device features are then determined for selected device 108. For many of the predicate features, the feature is dependent on the number of hops from selected device 108 to a predicate device. For example, there is one feature for the number of recalled devices that are one hop from selected device 108, a second feature for the number of recalled devices that are two or less hops from selected device 108 and a third feature for the number of recalled devices that are three or less hops from selected device 108. A summary of the features that are constructed is reported in Table 1, with the number of features that are created shown in parentheses. Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis by including accessing at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories and the risk is associated with a product recall and the categories are product categories as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
As per Claim 2 Leidner teaches the system of claim 1, wherein, to train the ML model, the training module is to:
assign, based on the general risk indicator dictionary and specific risk indicator dictionary, a risk score for each of the general risk indicators and the specific risk indicators, the risk score being based on a degree of similarity between the corresponding risk indicator and the one or more reasons for of each of the plurality of categories; wherein the training module is further configured to, on providing a description of the product to the ML model, receive a risk score from the ML model,. Leidner [0077] The ERPGS 1000 of FIG. 15 includes risk scoring and ERP generating module 1250 adapted to process news/media information received as input via news/media corpus 1100 and to identify risks associated with particular entities and arrive at risk scoring in processing news/media items related to one or more companies. ERP and risk score may be derived from computational linguistics and define or represent credible statements identified from, e.g., an article. The risk, as discussed in more detail below, will be interpreted as either positive, negative or neutral, and assigned respective polarizations, e.g., scores of +1, -1, and 0. The score may be derived from text and/or metadata from news/media and may apply a predefined or learned lexicon-based risk taxonomy or pattern to the processed text/metadata. The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
Leidner does not teaches wherein the risk score is indicative of a possibility of the product to get recalled However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis wherein the risk score is indicative of a possibility of the product to get recalled as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
As per Claim 3 Leidner teaches the system of claim 1, wherein the training module incorporates the ML model, the ML model being configured to, upon receiving a description of an entity, generate a set of risk scores for the product, wherein each risk score corresponds to a general risk indicator or a specific risk indicator. Leidner para. 94 teaches by creating an ERP based on perceived risks appearing in media and other resources, the present invention allows investment managers, industry analysts and chief risk officers to work with an ERP representative of a composite view taking into account all of the information that otherwise may be presented in the form of multiple alerts. With exemplary reference to FIG. 18, the ERP represents the company-specific risk profile and provides a more efficient "quick reference" which analysts may consider in making decisions. In one manner, the ERP is essentially a data structure based upon linguistic analysis wherein the data structure comprises risk parts or components representing risks associated with a company, e.g., Microsoft referenced at 2002. In this example, the four parts are: 1) a set of "General" risks (a set of <risk type; risk exposure indicator>pairs for a set of risk types that are applicable to all companies), referenced at 2004; 2) a set of "Idiosyncratic" risks (a set of <risk type; risk exposure indicator> pairs for a set of risk types that characterize particularly the company under consideration) referenced at 2006; 3) self trends (a set of historic signals and a forecasting trend that relates the company under consideration to its past overall risk exposure); and 4) peer trends (a set of historic signals and a forecasting trend that relates the company under consideration to the past overall risk exposure of its industry peers). The self trends and peer trends are referenced collectively at "Trends" at reference 2008. [0120] FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system. Counts may be normalized (e.g., log scale, frequency/popularity, or normalization by means of division be a "normal" value) and smoothed (e.g., autoregressive moving average (ARMA or .sigma.ARMA) model, autoregressive integrated moving average (ARIMA) model, exponential or Gaussian smoothing). In one manner, the system may determine that the term "currency" represents a negative risk in four instances and represents a positive risk in one instance. In that scenario, the list 2202 may include "currency" twice, once as a negative risk with a score of -4.0 and once as a positive risk with a score of +1.0. In this manner, a financial analyst may separately consider and review only negative risks and/or only positive risks. Although threats and opportunities are preferable processed and expressed separately, they may be compiled collectively. For instance, in one alternative the list may include the term "currency" only once with a composite score for that term of -3.0 (-4.0+1.0=-3.0). Again, an analyst can still review negative and positive risks; however, in this second scenario the term "currency" would only appear as a negative risk with a score of -3.0 rather than a score of -4.0. Scores or scoring may be normalized using any of a number of known methods. This example illustrates how using a multi-component risk profile provides greater analytical robustness and versatility to assist the user, e.g., financial analyst or risk manager, in decision making processes, e.g., investment decisions or managing corporate risk.
Leidner does not teaches the entity is a product However, Sen paras. 49 teaches feature construction 103 uses a selected device 108, referred to as the focal device, and predicate database 120, device recall database 123, device adverse events database 122 and device clearance records 124 to construct a set of features that can be applied to the predictive model. These features are divided into predicate features that are derived from attributes of predicate devices of a device and stand alone features that are derived only from attributes of the device. Further, some of the features are static features (time-invariant in that they do not change from year to year, while other features are time-varying. Time-varying features are computed by year. Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis for a particular product as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
As per Claim 5 Leidner teaches the system of claim 1, wherein the data retrieval module is further configured to periodically obtain updates to the data from the at least one product database. Leidner par. 117 teaches The ERP system takes into account trends and other historical information. The ERP system may use weighting techniques in one or more of its process. For instance, historical correlation between risks and stock movement may result in greater weight given to that correlation. Also, the ERP system may employ a "decay" factor, i.e., more recent mentions or risks are given more weight and older risks are given less weight. Also, can look to correlation between actual stock price movement and risk evaluation over time. Time theories, risk signals going up and down versus actual stock movement data. ERP risks may be compiled as if in periodic, e.g., daily, buckets, but can be milliseconds, seconds, hours, etc. Self trend is preferably a number on a particular day.
Leidner does not teach the data is recall data and the database is a recall database However, Sen para. 22 teaches the systems and methods described in the present disclosure provide several advantages over existing systems for monitoring medical device recalls. As one example, the disclosed systems and methods provide analytical insights within predicate device and device recall databases that are created from data extracted from regulatory authority data sources. For instance, the predicted probabilities of medical device recalls across different time windows (e.g., probability of recalls in 2-year or 3-year time windows) can be estimated. This insight is advantageous for medical device manufacturers and regulatory bodies as they consider marketing new medical devices through the 510(k) clearance pathway, or monitoring the chances of recalls for existing medical devices. As another example, the disclosed systems and methods create a predicate network for each medical device (including its predecessors and successors) that not only presents the citing relationship among devices but also shows which devices are recalled or have a high recall probability. The shortest citing path (in terms of network hops and/or approval year gap) from the focal device to the recalled device can also be computed and visualized to help evaluate the recall probability of the focal device. As yet another example, the disclosed systems and methods provide database creation and visualization processes that are automatic without requiring manual data scraping from data sources websites, making the analysis and visualization tasks scalable. Further, para. 31 teaches Data collection and extraction 102 includes steps and components for constructing and/or updating a predicate device database 120, constructing and/or updating a device adverse events database 122, constructing and/or updating a recall database 123, constructing and/or updating one or more device clearance records 124 that may contain additional medical device features retrieved from other data sources. Adverse events database 122 includes adverse events such as deaths and injuries associated with all 510(k) devices including the number of injuries, the number of deaths and the number of malfunctions associated with each device. Recall database 123 contains the recall history (e.g. recall date, resolving date, recall type, current recall status) for each device that has been recalled. Device clearance records 124 include basic device information such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices.
Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis the data is recall data and the database is a recall database as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
As per Claim 9 Leidner teaches method for predicting risk of recall of a product, comprising:
accessing one or more database to obtain data pertaining to a plurality of categories, the data comprising one or more reasons relating to each of the plurality of categories; such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices. Leidner para. 75 teaches the ERPGS may further process information, including historical trading information, historical risk information, and historical ERP and risk scores to arrive at an anticipated or predictive behavior of stock price and other investment vehicles. The ERPGS 1000 leverages traditional and new media resources to provide a risk-based solution that expands the scope of conventional tools to provide an enhanced analysis data structure for use by financial analysts, investment managers, risk managers and others. Para. 77 teaches The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
obtaining a general risk indicator dictionary comprising a set of general risk indicators corresponding to the plurality of categories, wherein a general risk indicator is indicative of a user-defined criteria for assessing risk of the plurality of categories; and Leidner para. 45 teaches These generated patterns can be used for re-processing the corpus, in one embodiment of the present invention after some human review, or automatically in another embodiment. The extracted sentences or sentence sequences are then both validated (whether or not they are really risk-indicating sentences) and parsed into risks of the form P=>Q (i.e. finding out which text spans correspond to the precondition "P", which parts express the implication "=>", and which parts express the high-impact event "Q"), using, but not limited to, the following non-limiting features: a set of terms with significant statistical association with the term "risk" (in one embodiment of this invention, statistical programs, such as Pointwise Mutual Information (PMI) and Log Likelihood, or rules, including but not limited to rules obtained by Hearst pattern induction, may be used to determine the set of terms); a set of binary gazetteer features, where the feature fires if a gazetteer a set of risk-indicative terms ("threat", "bankruptcy", "risk", . . . ) compiled by human experts or extracted from hand-labeled training data; a set of indicators of speculative language; instances of future time reference; occurrences of conditionals; and/or occurrences of causality markers.
Leidner para. 90 teaches In one exemplary method of the present invention, and with reference to FIG. 17A, a method for generating an ERP 3000 is illustrated as follows. Initially, at step 3010, the ERP system obtains information and content of interest from credible news/media sources (news feeds, blogs, websites, etc.) from internal or external sources. At step 3020 the ERP system applies linguistic computational analysis and learned risk taxonomy to the obtained information from step 3010 to identify risks and entities referenced or mentioned in the content. At step 3030, the ERP system associates risks with one or more entities identified in step 3020. At step 3040, the system applies a risk taxonomy to arrive at a separate score or indication or a derivative score or indication for at least two risk components: General or generic risks; idiosyncratic risks; self-trend; and peer trend. [0119] Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high.
obtaining a specific risk indicator dictionary comprising a set of specific risk indicators corresponding to each category from amongst the plurality of categories, wherein a specific risk indicator is indicative of a user-defined criteria for assessing risk of the corresponding category, Leidner para. 120 teaches FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system.
wherein the general risk indicator dictionary and specific risk indicator dictionary include a list of keywords for each of the general risk indicators and specific risk indicators, respectively, a keyword being a term semantically similar to the corresponding risk indicator; Leidner para. 119-120 teaches Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high. FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system.
training a Machine-Learning (ML) model, based on the data, for predicting risk, wherein the training comprises: assigning, based on the general risk indicator dictionary and specific risk indicator dictionary, a risk score for each of the general risk indicators and the specific risk indicators, the risk score being based on a degree of similarity between the corresponding risk indicator and the one or more reasons of each of the plurality of product categories; and on providing a description to the ML model, receiving a risk score from the ML model, wherein the risk score is indicative of the risk. Leidner para. 48 teaches A risk clusterer 240 groups all risks in the risk database by similarity, but without imposing a pre-defined taxonomy (data driven). In one embodiment Hearst pattern induction may be used. Hearst pattern induction was first mentioned in Hearst, Marti, "WordNet: An Electronic Lexical Database and Some of its Applications", (Christiane Fellbaum (Ed.)), MIT Press 1998, the contents of which is incorporated herein by reference. In another embodiment of the present invention a number k is chosen by the system developer, and the kNN-means clustering method may be used. Further details of kNN clustering is described by Hastie, Trevor, Robert Tibshirani and Jerome Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Second Edition Springer (2009), the content of which is incorporated herein by reference. In such a case, the risks are grouped into a number, i.e. k, of categories and then classified by choosing the cluster with the highest similarity to a cluster of interest. In another embodiment of the present invention, hierarchical clustering is used. Alternatively or in addition to, both k-means clustering and hierarchical clustering may be used. Further para. 77 teaches The ERPGS 1000 of FIG. 15 includes risk scoring and ERP generating module 1250 adapted to process news/media information received as input via news/media corpus 1100 and to identify risks associated with particular entities and arrive at risk scoring in processing news/media items related to one or more companies. ERP and risk score may be derived from computational linguistics and define or represent credible statements identified from, e.g., an article. The risk, as discussed in more detail below, will be interpreted as either positive, negative or neutral, and assigned respective polarizations, e.g., scores of +1, -1, and 0. The score may be derived from text and/or metadata from news/media and may apply a predefined or learned lexicon-based risk taxonomy or pattern to the processed text/metadata. The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
Leidner does not teach access at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories However, Sen para. 22 teaches The systems and methods described in the present disclosure provide several advantages over existing systems for monitoring medical device recalls. As one example, the disclosed systems and methods provide analytical insights within predicate device and device recall databases that are created from data extracted from regulatory authority data sources. For instance, the predicted probabilities of medical device recalls across different time windows (e.g., probability of recalls in 2-year or 3-year time windows) can be estimated. This insight is advantageous for medical device manufacturers and regulatory bodies as they consider marketing new medical devices through the 510(k) clearance pathway, or monitoring the chances of recalls for existing medical devices. As another example, the disclosed systems and methods create a predicate network for each medical device (including its predecessors and successors) that not only presents the citing relationship among devices but also shows which devices are recalled or have a high recall probability. The shortest citing path (in terms of network hops and/or approval year gap) from the focal device to the recalled device can also be computed and visualized to help evaluate the recall probability of the focal device. As yet another example, the disclosed systems and methods provide database creation and visualization processes that are automatic without requiring manual data scraping from data sources websites, making the analysis and visualization tasks scalable. Further para. 31 teaches [0031] Data collection and extraction 102 includes steps and components for constructing and/or updating a predicate device database 120, constructing and/or updating a device adverse events database 122, constructing and/or updating a recall database 123, constructing and/or updating one or more device clearance records 124 that may contain additional medical device features retrieved from other data sources. Adverse events database 122 includes adverse events such as deaths and injuries associated with all 510(k) devices including the number of injuries, the number of deaths and the number of malfunctions associated with each device. Recall database 123 contains the recall history (e.g. recall date, resolving date, recall type, current recall status) for each device that has been recalled. Device clearance records 124 include basic device information such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices.
Leidner does not teach the analyzed risk is associated with a product recall and the categories are product categories. However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Further par. 49-51 teach [0049] Feature construction 103 uses a selected device 108, referred to as the focal device, and predicate database 120, device recall database 123, device adverse events database 122 and device clearance records 124 to construct a set of features that can be applied to the predictive model. These features are divided into predicate features that are derived from attributes of predicate devices of a device and stand alone features that are derived only from attributes of the device. Further, some of the features are static features (time-invariant in that they do not change from year to year, while other features are time-varying. Time-varying features are computed by year.
To form the predicate features, a predicate network 130 is formed for selected device 108 using predicate database 120. In predicate network 130, devices that are connected directly to selected device 108 are said to be one hop from selected device 108, devices that are connected to selected device 108 through one intermediary device are said to be two hops from selected device 108 and devices that are connected to device 108 through two intermediary devices are said to be three hops from selected device 108. If there are two paths between a device and selected device 108, the shortest path is used.
Predicate features and device features are then determined for selected device 108. For many of the predicate features, the feature is dependent on the number of hops from selected device 108 to a predicate device. For example, there is one feature for the number of recalled devices that are one hop from selected device 108, a second feature for the number of recalled devices that are two or less hops from selected device 108 and a third feature for the number of recalled devices that are three or less hops from selected device 108. A summary of the features that are constructed is reported in Table 1, with the number of features that are created shown in parentheses.
wherein the risk score is indicative of a possibility of the product being recalled However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis by including accessing at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories; the risk is associated with a product recall and the categories are product categories and the risk score indicating a possibility of the new product being recalled as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
As per Claim 15 Leidner teaches A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: (see para. 42).
access data pertaining to a plurality of categories from at least one database, the data comprising one or more reasons for of each of the plurality of categories; such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices. Leidner para. 75 teaches the ERPGS may further process information, including historical trading information, historical risk information, and historical ERP and risk scores to arrive at an anticipated or predictive behavior of stock price and other investment vehicles. The ERPGS 1000 leverages traditional and new media resources to provide a risk-based solution that expands the scope of conventional tools to provide an enhanced analysis data structure for use by financial analysts, investment managers, risk managers and others. Para. 77 teaches The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
receive user input defining general risk indicators corresponding to the plurality of categories, wherein a general risk indicator is indicative of a user-defined criteria for assessing risk applicable across the plurality of categories, and Leidner para. 45 teaches These generated patterns can be used for re-processing the corpus, in one embodiment of the present invention after some human review, or automatically in another embodiment. The extracted sentences or sentence sequences are then both validated (whether or not they are really risk-indicating sentences) and parsed into risks of the form P=>Q (i.e. finding out which text spans correspond to the precondition "P", which parts express the implication "=>", and which parts express the high-impact event "Q"), using, but not limited to, the following non-limiting features: a set of terms with significant statistical association with the term "risk" (in one embodiment of this invention, statistical programs, such as Pointwise Mutual Information (PMI) and Log Likelihood, or rules, including but not limited to rules obtained by Hearst pattern induction, may be used to determine the set of terms); a set of binary gazetteer features, where the feature fires if a gazetteer a set of risk-indicative terms ("threat", "bankruptcy", "risk", . . . ) compiled by human experts or extracted from hand-labeled training data; a set of indicators of speculative language; instances of future time reference; occurrences of conditionals; and/or occurrences of causality markers.
Leidner para. 90 teaches In one exemplary method of the present invention, and with reference to FIG. 17A, a method for generating an ERP 3000 is illustrated as follows. Initially, at step 3010, the ERP system obtains information and content of interest from credible news/media sources (news feeds, blogs, websites, etc.) from internal or external sources. At step 3020 the ERP system applies linguistic computational analysis and learned risk taxonomy to the obtained information from step 3010 to identify risks and entities referenced or mentioned in the content. At step 3030, the ERP system associates risks with one or more entities identified in step 3020. At step 3040, the system applies a risk taxonomy to arrive at a separate score or indication or a derivative score or indication for at least two risk components: General or generic risks; idiosyncratic risks; self-trend; and peer trend. [0119] Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high.
specific risk indicators corresponding to each category from amongst the plurality of categories, wherein a specific risk indicator is indicative of a user-defined criteria for assessing risk of the corresponding category;, Leidner para. 120 teaches FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system.
generate lists of semantically similar keywords for each of the general risk indicators and specific risk indicators; create a general risk indicator dictionary and a specific risk indicator dictionary by grouping each of the general risk indicators with their corresponding list of keywords and each of the specific risk indicators with their corresponding list of keywords, respectively; Leidner para. 119-120 teaches Now with reference to the graphical representation of FIG. 19, General risks 2004 may be comprised of financial risks 2102, operational risks 2104, legal risks 2106, and market risks 2108, and may be represented in scalar form or normalized as desired. General Risks are those risks that are designated to be universal, i.e., they apply to all companies--financial, operational, legal, and market. Typically, all business concerns are exposed to these general risk types. For other types of entities, the defined set of general risk types may be tailored to best represent that entity type, e.g., political figure. In this case, because all companies have these risk types, they get mentioned often, and therefore counts are quite high. FIG. 20 represents an exemplary set of Idiosyncratic Risks 2006, which generally represent all other non-general or more specific risks. In this example, the set of risks 2202 represent text terms identified and extracted through the ERP process from content sources as being reliable and as being associated with a specific entity and representing risk associated with that entity. In this example, scores 2204 may represent each instance of the idiosyncratic risk mentioned in a content piece. The set 2202 includes the terms "bad debts", "permanent change", "currency", "higher interest rates", "super injunctions", etc. In this example, a score is assigned to each idiosyncratic risk, e.g., "bad debts" 2206 received a score of -1.0 2208, and "currency" 2210 received a score of -3.0 2212. The scores may be based on a count of the terms that appear in a corpus of content, e.g., the term currency appeared three times in one or more article, press release, regulatory filing, legal document, or other unit of content included in the corpus or set of content processed by the ERP generating system.
provide, to a machine learning (ML) model, the general risk indicator dictionary and the specific risk indicator dictionary along with the data as training data to train the ML model to predict risk, wherein the ML model is to determine, based on the general risk indicator dictionary and specific risk indicator dictionary, a degree of similarity between each of the general risk indicators and the specific risk indicators and the one or more reasons of each of the plurality of categories; obtain, from the ML model, in response to providing attributes of a new entity to the ML model, a risk score for the new entity,. Leidner para. 48 teaches A risk clusterer 240 groups all risks in the risk database by similarity, but without imposing a pre-defined taxonomy (data driven). In one embodiment Hearst pattern induction may be used. Hearst pattern induction was first mentioned in Hearst, Marti, "WordNet: An Electronic Lexical Database and Some of its Applications", (Christiane Fellbaum (Ed.)), MIT Press 1998, the contents of which is incorporated herein by reference. In another embodiment of the present invention a number k is chosen by the system developer, and the kNN-means clustering method may be used. Further details of kNN clustering is described by Hastie, Trevor, Robert Tibshirani and Jerome Friedman, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", Second Edition Springer (2009), the content of which is incorporated herein by reference. In such a case, the risks are grouped into a number, i.e. k, of categories and then classified by choosing the cluster with the highest similarity to a cluster of interest. In another embodiment of the present invention, hierarchical clustering is used. Alternatively or in addition to, both k-means clustering and hierarchical clustering may be used. Further,
para. 77 teaches The ERPGS 1000 of FIG. 15 includes risk scoring and ERP generating module 1250 adapted to process news/media information received as input via news/media corpus 1100 and to identify risks associated with particular entities and arrive at risk scoring in processing news/media items related to one or more companies. ERP and risk score may be derived from computational linguistics and define or represent credible statements identified from, e.g., an article. The risk, as discussed in more detail below, will be interpreted as either positive, negative or neutral, and assigned respective polarizations, e.g., scores of +1, -1, and 0. The score may be derived from text and/or metadata from news/media and may apply a predefined or learned lexicon-based risk taxonomy or pattern to the processed text/metadata. The ERPGS 1000 may include a training or learning module 1270 that analyzes past or archived news/media, and may include use of a known training set of data, and may consider historical stock price information, especially in comparison with historical "facts" or events. In this manner the ERPGS may be adapted to build a model to predict stock behavior given certain types of news or events.
Leidner does not teach access at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories However, Sen para. 22 teaches The systems and methods described in the present disclosure provide several advantages over existing systems for monitoring medical device recalls. As one example, the disclosed systems and methods provide analytical insights within predicate device and device recall databases that are created from data extracted from regulatory authority data sources. For instance, the predicted probabilities of medical device recalls across different time windows (e.g., probability of recalls in 2-year or 3-year time windows) can be estimated. This insight is advantageous for medical device manufacturers and regulatory bodies as they consider marketing new medical devices through the 510(k) clearance pathway, or monitoring the chances of recalls for existing medical devices. As another example, the disclosed systems and methods create a predicate network for each medical device (including its predecessors and successors) that not only presents the citing relationship among devices but also shows which devices are recalled or have a high recall probability. The shortest citing path (in terms of network hops and/or approval year gap) from the focal device to the recalled device can also be computed and visualized to help evaluate the recall probability of the focal device. As yet another example, the disclosed systems and methods provide database creation and visualization processes that are automatic without requiring manual data scraping from data sources websites, making the analysis and visualization tasks scalable. Further para. 31 teaches [0031] Data collection and extraction 102 includes steps and components for constructing and/or updating a predicate device database 120, constructing and/or updating a device adverse events database 122, constructing and/or updating a recall database 123, constructing and/or updating one or more device clearance records 124 that may contain additional medical device features retrieved from other data sources. Adverse events database 122 includes adverse events such as deaths and injuries associated with all 510(k) devices including the number of injuries, the number of deaths and the number of malfunctions associated with each device. Recall database 123 contains the recall history (e.g. recall date, resolving date, recall type, current recall status) for each device that has been recalled. Device clearance records 124 include basic device information such as device approval date, manufacturer, product code, medical specialty of all 510(k) devices.
Leidner does not teach the analyzed risk is associated with a product recall and the categories are product categories. However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Further par. 49-51 teach [0049] Feature construction 103 uses a selected device 108, referred to as the focal device, and predicate database 120, device recall database 123, device adverse events database 122 and device clearance records 124 to construct a set of features that can be applied to the predictive model. These features are divided into predicate features that are derived from attributes of predicate devices of a device and stand alone features that are derived only from attributes of the device. Further, some of the features are static features (time-invariant in that they do not change from year to year, while other features are time-varying. Time-varying features are computed by year.
To form the predicate features, a predicate network 130 is formed for selected device 108 using predicate database 120. In predicate network 130, devices that are connected directly to selected device 108 are said to be one hop from selected device 108, devices that are connected to selected device 108 through one intermediary device are said to be two hops from selected device 108 and devices that are connected to device 108 through two intermediary devices are said to be three hops from selected device 108. If there are two paths between a device and selected device 108, the shortest path is used.
Predicate features and device features are then determined for selected device 108. For many of the predicate features, the feature is dependent on the number of hops from selected device 108 to a predicate device. For example, there is one feature for the number of recalled devices that are one hop from selected device 108, a second feature for the number of recalled devices that are two or less hops from selected device 108 and a third feature for the number of recalled devices that are three or less hops from selected device 108. A summary of the features that are constructed is reported in Table 1, with the number of features that are created shown in parentheses.
the risk score indicating a possibility of the new product being recalled However, Sen Abstract teaches a system for estimating a recall probability for a medical device includes a predicate device database having stored thereon relationships between a plurality of medical devices. A processor is in communication with the predicate device database and is configured to generate a network of medical devices having a relationship to a focal medical device using the predicate device database. The generated network is used to form features, which are applied to a predictive model to determine the recall probability. Both Leidner and Sen are directed to parsing unstructured data to identify risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to perform recall risk analysis by including accessing at least one product recall database to obtain recall data pertaining to a plurality of categories of products, the recall data comprising one or more reasons for recall relating to each of the plurality of product categories; the risk is associated with a product recall and the categories are product categories and the risk score indicating a possibility of the new product being recalled as taught by Sen to use the risk analysis system of Leidner to analyze other risks including product recall risk. The result is a more flexible and useful system. Further, one of ordinary skill in the art before the effective filing date of the Applicant’s invention it would have recognized that applying the known technique of Sen would have yielded predictable results and resulted in an improved system. It would have been recognized that applying the technique of Sen to the teachings of Leidner would have yielded predictable results because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate product recall risk prediction into similar risk prediction systems. Further, incorporating the recall risk prediction taught by Sen to the system taught by Leidner would result in an improved system that provides an efficient, flexible and accurate means to analyze large amounts of data to accurately predict recalls.
Claim(s) 10 recite(s) similar limitation to those recited in Claim 3 and is/are rejected for similar reasons. Further Leidner teaches a method to perform the recited steps. (see Leidner Abstract)
Claim(s) 16, 20 recited similar limitation to those recited in Claim 3, 5 and is/are rejected for similar reasons. Further Leidner teaches A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the recited steps (see para. 42).
Claim(s) 4, 11, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leidner US 20120221485 A1 in view of Sen US 2024/0233932 A1 as applied to Claim 1 and in further view of Manapat US 2023/0230090 A1.
As per Claim 4 Leidner does not teach the system of claim 3, wherein the ML model is further configured to identify the general risk indicator or specific risk indicator associated with a highest risk score amongst the set of risk scores as a primary cause for recall of the product. However, Manapat [0181] teaches According to another embodiment of method 900, transmitting the risk explanation to the merchant responsive to the inquiry includes transmitting a single primary risk factor to the user as the risk explanation responsive to the inquiry; in which the single primary risk factor corresponds to a predicate derived from the risk model that when input into the risk model results in a simulated fraud likelihood score having a highest correlation to the fraud likelihood score output by the risk model for the one purchase transaction for which the risk explanation is requested by the user. Both Leidner in view of Sen and Manapat are directed to assessing risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner in view of Sen to include identify risk indicator associated with a highest risk score amongst the set of risk scores as a primary cause as taught by Manapat to clearly communicate the most likely cause of the risk to be displayed to a user.
Claim(s) 11 recite(s) similar limitation to those recited in Claim 4 and is/are rejected for similar reasons. Further Leidner teaches a method to perform the recited steps. (see Leidner Abstract)
Claim(s) 17 recited similar limitation to those recited in Claim 4 and is/are rejected for similar reasons. Further Leidner teaches A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the recited steps (see para. 42).
Claim(s) 6, 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leidner US 20120221485 A1 in view of Sen US 2024/0233932 A1 as applied to Claim 1 and in further view of Bianchini US 2025/0139151 A1.
As per Claim 6 Leidner teaches the system of claim 1, wherein the generative module uses a model to generate the list of keywords for each of the general risk indicators and specific risk indicators. Leidner para. 49 teaches FIG. 4 depicts one embodiment of the risk clusterer 240 according to the present invention. At step 310, a text corpus is provided. At step 320, the text corpus is tokenized into a set of sentences. At step 330, all instances of a risk, which is indicated by "*", is extracted from the tokenized text. At step 340, a taxonomy of risks is constructed into a tree by organizing all fillers matching the risk, i.e."*". At step, 350, Hearst pattern induction may be used to induce the risk taxonomy. Further, an NP chunker may be used to find the boundaries of interest.
Leidner does not teach using a Large Language Model (LLM) However, Bianchini [0060] teaches The large language model 160 can process textual input, user identifiers, and the state of the electronic report 130 to generate corresponding outputs. The large language model 160 can perform input analysis by identifying relevant keywords from the textual input, evaluating the context within the electronic report 130, and referencing user preferences or interaction history. The large language model 160 can evaluate the textual input to determine the underlying intent of a user, such as the specific action the user seeks to accomplish within the electronic report 130. Based on the analysis, the large language model 160 can generate one or more outputs, such as a list of keywords sorted by relevance, a more objective rephrased version of the textual input, and an intent sentence that provides a summary of the underlying report intent. For example, if the textual input is “show employees with salaries above $50,000”, the large language model 160 can generate a keyword list including “employees”, “salaries”, and “above $50,000”, a rephrased input such as “display employees earning more than $50,000”, and an intent sentence such as “filter employee list by salary threshold”. Both Leidner and Bianchini are directed to parsing ab unstructured corpus. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner to include using a Large Language Model (LLM) as taught by Bianchini to improve the relevance and accuracy of the generated output (see para. 5).
Claim(s) 12 recite(s) similar limitation to those recited in Claim 6 and is/are rejected for similar reasons. Further Leidner teaches a method to perform the recited steps. (see Leidner Abstract)
Claim(s) 7, 13, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leidner US 20120221485 A1 in view of Sen US 2024/0233932 A1 as applied to Claim 1 and in further view of Keibler US 2023/0289914 A1.
As per Claim 7 Leidner teaches the system of claim 1, further comprising a recommendation module coupled to the one or more processors, the recommendation module being configured to generate, based on the risk score, one or more recommendations to prevent and mitigate the predicted risk of recall of the product. However, Keibler [0037] In some implementations, the risk management and mitigation tools 108 identify potential risk areas and provide suggestions for mitigating or avoiding the risk. The risk management and mitigation tools 108, for example, may include pricing tools for appropriately pricing insurance based upon calculated entity risk within subject supply chains. Further, the risk management and mitigation tools 108 may include identification of a “weakest link” in a supply chain (e.g., worst scoring producer 112b) and recommend removal or replacement of the problem supplier. In another example, the risk management and mitigation tools 108 may identify a shipping problem (e.g., temperature abuse) that could be resolved by adjusting shipping, adjusting production, and/or adjusting recipe. Both Leidner in view of Sen and Keibler are directed to identifying risk. Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the Applicant’s invention to modify the teachings of Leidner in view of Sen to a recommendation module coupled to the one or more processors, the recommendation module being configured to generate, based on the risk score, one or more recommendations to prevent and mitigate the predicted risk of recall of the product as taught by Keibler to enabling both companies and their suppliers to better understand critical threats, explore methods to control these risks, and identify products and/or services to protect against potential business-threatening financial and reputational liabilities (see para. 5).
Claim(s) 13 recite(s) similar limitation to those recited in Claim 7 and is/are rejected for similar reasons. Further Leidner teaches a method to perform the recited steps. (see Leidner Abstract)
Claim(s) 18 recited similar limitation to those recited in Claim 7 and is/are rejected for similar reasons. Further Leidner teaches A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the recited steps (see para. 42).
Claim(s) 8, 14, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leidner US 20120221485 A1 in view of Sen US 2024/0233932 A1 as applied to Claim 1 and in further view of Keibler US 2023/0289914 A1 as applied to claims 7, 13, 18 and in further view of Altakrouri US 2025/0139581 A1.
As per Claim 8 Leidner teaches 8. The system of claim 7, wherein the one or more recommendations are generated using a Large Language Model (LLM) trained on historic data comprising records of actions taken regarding previously recalled products. However, Altakrouri [0040] teaches FIG. 3A shows a workflow (300) of the construction control system (160) for generating and presenting the contractor improvisation index. In step (301), a certain contract is selected to be observed for improvisation actions by the contractor. For example, the contract is an agreement document (e.g., paper document or digital document) for the construction project to be performed by the contractor. In step (302), a Machine Learning (ML) module within the construction control system (160) for calculating the index is activated and optimized based on the type or other characteristics of the selected contract, planned activities, hazard plan and implementation timeline. For example, the ML module may include software and/or hardware components that execute ML algorithms. This step relies on a Models Space (303) within the construction control system (160) that is built on top of five ML models each covers an important aspect related to the improvisation index. For example, the Models Space (303) may be a data repository included in or accessible by the construction control system (160). The Incidents Model (304) is a ML model that is trained on historical incident data (305) relating to prior minor safety incidents, major safety incidents, safety observations, improvisation actions, etc. of the contractor. The Contextual Model (306) is a ML model that is trained on historical contextual data (307) based on the digital twin (i.e., a digital representation) and IoT data of previous related construction projects. IoT data may include data about the health of heavy machinery, workers locations, workers flow of movements, construction status, etc. The Fine-Tuned Large Language Model (308) is a Large Language Model (LLM) that is fine-tuned based on a large set of documents (309) related to projects, contracts and other documents available inside or outside the organization knowledge base such as job safety analysis documents, safety related articles, safety briefing documents, etc. of the selected contract. The Process Minor (310) model models the construction projects' processes and executional paths (311). The Recommendation Model (312) is an ML model that is trained on historical and general instruction data relating to prior incidents mitigations and control measures (313). The activated ML models collectively form the core intelligence for all monitoring activities recorded in the monitoring space (314), which is a data repository included in or accessible by the construction control system (160). The recorded monitoring data in the monitor space (314) relates to monitoring for incidence likelihood (315), process deviations (316), contract events (317) and workers activities (318).
Claim(s) 14 recite(s) similar limitation to those recited in Claim 8 and is/are rejected for similar reasons. Further Leidner teaches a method to perform the recited steps. (see Leidner Abstract)
Claim(s) 19 recited similar limitation to those recited in Claim 8 and is/are rejected for similar reasons. Further Leidner teaches a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the recited steps (see para. 42).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEIRDRE D HATCHER whose telephone number is (571)270-5321. The examiner can normally be reached Monday-Friday 8-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached at 571-270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEIRDRE D HATCHER/Primary Examiner, Art Unit 3625
Prosecution Insights