Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This communication is a Final Office Action in response to Applicant’s amendment for application number 18/673,378 received on 04/01/2026.
In accordance with Applicant’s amendment, claims 1-23 are amended, currently pending, and have been examined.
Response to Amendment
The amendment filed on 04/01/2026 has been entered.
Applicant’s amendment necessitated the new ground(s) of rejection set forth in this Office Action.
Upon review of amended claims, the claim objections previously applied are withdrawn.
Upon review of amended claims, the 112(b) rejections previously applied are withdrawn.
Response to Arguments
Response to §101 arguments – Applicant’s arguments with respect to the §101 rejections previously applied to the claims have been considered and are unpersuasive.
Applicant argues (Remarks at pg. 12): “These limitations are analogous to the claims in Research Corp. Techs. v. Microsoft Corp., 627 F.3d 859, 868, 97 USPQ2d 1274, 1280 (Fed. Cir. 2010), which the MPEP identifies as not reciting mental processes because they "required the manipulation of computer data structures (e.g., the pixels of a digital image and a two-dimensional array known as a mask) and the output of a modified computer data structure." Similarly, the claimed manipulation of vector embeddings and similarity-based retrieval from a vector database requires manipulation of computer data structures that cannot practically be performed mentally.”. In response, Examiner respectfully disagrees and notes that the present claims do not provide an analogous improvement to the computer to that of RCT. The present claims are not directed to halftoning gray scale images. Therefore, Examiner respectfully disagrees with Applicant's assertion that the present claims are directed to statutory subject matter in view of RCT.
Applicant argues (Remarks at pg. 13): “With respect to the Examiner's classification of the claims under "Certain Methods of Organizing Human Activity", specifically citing "managing personal behavior or relationships or interactions between people", the amended claims describe machine-to-machine coordination, not human-to-human interaction. The task planner dispatches structured plans to the executor, which in turn coordinates specialized software agents (generator, transformer, summarizer). No human manages these relationships; they are automated software orchestration steps executed by defined system components. The "collecting context by querying the user" step, when considered in the context of the claims as a whole, is part of a technical data collection pipeline that feeds into the vector database storage and RAG-based sufficiency determination. It does not transform the entire claim into one directed to organizing human activity.”. In response, Examiner respectfully disagrees and notes that the limitation for “collecting context relevant to the query based on availability of internal information retrieval by querying the user with one or more clarification questions and receiving responses” falls under the Certain Methods of Organizing Human activity abstract idea grouping directed to managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) because it requires a user to answer questions (i.e., following instructions).
Applicant argues (Remarks at pg. 14): “Second, the amended claims are analogous to claims found eligible under the "improvements" consideration. In Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1339 (Fed. Cir. 2016), claims to a self-referential table for a computer database were directed to an improvement in computer capabilities because the specification discussed how the invention improved the way the computer stores and retrieves data in memory in combination with the specific data structure recited in the claims. Similarly, the amended claims recite a specific data structure (vector embeddings) and a specific retrieval mechanism (RAG process) that improve how the system stores and retrieves market research data for sufficiency determinations.”. In response, Examiner respectfully disagrees and notes that the present claims do not provide an analogous improvement to the computer to that of Enfish, specifically because the present claims do not improve the computer itself. The improvements of a self-referential table provide a specific benefit to the functioning of the computer, which is not the case in the claims of the instant application. The present claims are directed to analyzing data to determine a response to a user query, which is not an improvement to the computer itself. Rather, this is an improvement to the abstract idea associated with “Mental Processes” and “Certain Methods of Organizing Human Activity”.
Applicant argues (Remarks at pgs. 14-15): “Even if the Examiner were to maintain that certain high-level steps (e.g., "forming a strategy," "determining market research parameters") could theoretically be characterized as mental processes in isolation, the claims must be evaluated "as a whole" and not by dissecting individual limitations. The Desjardins memorandum instructs that "Examiners and panels should not evaluate claims at such a high level of generality" that potentially meaningful technical limitations are dismissed without adequate explanation.”. In response, Examiner respectfully disagrees and notes that the present claims do not provide an analogous improvement to the machine learning model (e.g. reinforcement learning model). Examiner respectfully asserts that the claims are unlike the Desjardins decision because the claims are directed to an abstract idea versus being directed to an improvement to computer functionality. The present claims do not provide an analogous technical solution to that of Desjardins because the claims do not “address challenges in continual learning and model efficiency by reducing storage requirements and preserving task performance across sequential training.” For example, the machine learning model (including RAG) of the present claims is merely a tool to perform the abstract process. An improvement to the presented analysis, such as market research parameters and reports, would be an improvement to the abstract limitations for consideration under Step 2A, Prong 1 and not to the reinforcement learning model itself. MPEP 2106.05(a): “It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements...” Additionally, as discussed in 2106.05(a)(II) improvements to technology or technical fields, “an improvement in the abstract idea itself … is not an improvement in technology”
Applicant argues (Remarks at pg. 15): “These are improvements to the accuracy, completeness, and reliability of the information retrieval system's output, not merely faster execution of the same process. Cf. Enfish, 822 F.3d at 1335-36 (improvement to the way data is stored and retrieved in a computer is a technical improvement); Core Wireless Licensing S.A.R.L. v. LG Elecs., Inc., 880 F.3d 1356, 1362 (Fed. Cir. 2018) (specific way of displaying information that improved system functionality was eligible).”. In response, Examiner respectfully disagrees and notes that the present claims do not provide an analogous improvement to the computer to that of Core Wireless, specifically because the present claims do not improve the graphical user interface itself. The improvements of Core Wireless provide a specific benefit the user interface for electronic devices, which is not the case with the instant claims. The present claims are directed to analyzing data to determine a response to a user query, which is not an improvement to the computer itself. The claims do not improve a problem rooted in computer technology because the computer is being used as a tool to analyze data.
Response to §103 arguments – Applicant’s arguments with respect to the §103 rejections previously applied to the claims are considered moot based on the new ground(s) of rejection set forth in this Office Action as necessitated by the amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-patentable subject matter. The claims are directed to an abstract idea without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception as further set forth in MPEP 2106.
Step 1: The claimed invention is analyzed to determine if it falls outside one of the four statutory categories of invention. See MPEP 2106.03
Claim(s) 1-8, and 10-12 is/are directed to a method (i.e., Process), and claim(s) 9, and 13-19 is/are directed to a system (i.e., Machine), and claim(s) 20, and 21-23 is/are directed to a non-transitory computer-readable storage medium (i.e., Manufacture). Therefore, the claims are directed to patent eligible categories of invention. Accordingly, the claims satisfy Step 1 of the eligibility inquiry.
As drafted, the limitations recited by claims 1-23 fall under the “Mental Processes” abstract idea group by setting forth activities that could be performed mentally by a human (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III), and “Certain Methods of Organizing Human Activity” directed to managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (see MPEP § 2106.04(a)(2), subsection II).
Independent claim 1 recites a computer-implemented method for automating market research with the following abstract limitations: “collecting context relevant to the query based on availability of internal information retrieval by querying the user with one or more clarification questions and receiving responses; forming a strategy to retrieve data from available data sources based on the context, wherein the strategy formation comprises determining reliability of the available data sources and prioritizing the available data sources according to relevance to the research objective; and (ii) agent-based sources including at least one of synthetic focus groups, synthetic surveys, and autonomous survey collection, wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective; and analysing the data to determine if the retrieved data is sufficient to complete the research objective; analyzing the retrieved data to determine whether the retrieved data is sufficient to complete the research objective; requesting modification or additional information in the data based on the analysis, responsive to the retrieved data not being determined as sufficient to complete the research objective; determining market research parameters based on the analysed data, the market research parameters comprising dynamic reports with visualizations and/or structured reports based on predefined templates; and iteratively optimizing market research parameters by re-weighting reliability scores of the available data sources based on quality metrics derived from the retrieved data; that generates a step-by-step plan in response to the query“. But for the recitation of additional elements recited by the claim, the steps in the claim could be accomplished mentally, such as by human observation, evaluation, judgement, opinion, or with the help of pen and paper. Independent claim 9 recites a system with limitations that are substantially similar to the limitations of independent claim 1, therefore, the same analysis applies. Additionally, the limitation for “collecting context relevant to the query based on availability of internal information retrieval by querying the user with one or more clarification questions and receiving responses” also falls under the Certain Methods of Organizing Human Activity abstract idea grouping directed to managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions).
Dependent claims 10 and 11 further narrow the abstract idea and introduce further additional elements for consideration.
Dependent claims 2-8, 12-19, and 21-23 further narrow the abstract idea and do not introduce any additional elements for consideration.
Step 2A, Prong 2: An evaluation is made whether a claim recites any additional element, or combination of additional elements, that integrate the judicial exception into a practical application of the exception. See MPEP 2106.04(d).
Regarding the computing additional elements, namely a memory unit configured to store machine-readable instructions, a processor operably connected to the memory unit, and a non-transitory computer-readable storage medium from the independent claims, these additional elements have been evaluated but fail to integrate the abstract idea into a practical application because they amount to using generic computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (generic computing environment). See MPEP 2106.05(f) and 2106.05(h). In addition, these limitations fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, fail to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment (generic computing environment).
With respect to the limitations for via a user-proxy artificial intelligence (AI) agent executed by a multi-agent framework server of a market research system (MRS), using one or more Al agents within the MRS, the available data sources comprising: (i) non-agent sources including at least one of web scraping, Application Programming Interface (API) endpoints, web browsing, and uploaded files;, using a retrieval augmented generation (RAG) process that retrieves one or more of the embeddings from the vector database responsive to the query, and activating the one or more Al agents within the MRS based on the re-weighted reliability scores, incoming data and feedback, and wherein the multi-agent framework server implements a task planner, and an executor that sequentially implements steps of the plan by coordinating with specialized agents including at least one of a generator, a transformer and a summarizer from the independent claims, these limitations fail to integrate the abstract idea into a practical application because they provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception.
With respect to the limitations for receiving a query from a user within the MRS, wherein the query comprises a research objective and at least one of a standard report, and one or more questions, and retrieving data from the available data sources based on the strategy from the independent claims, these limitations do not integrate the judicial exception into a practical application because they add insignificant extra-solution activity to the judicial exception. This limitation merely recites receiving and/or sending data over a network, which is extra-solution activity. See MPEP 2106.05(g).
With respect to the limitations for storing at least a portion of the retrieved data in a vector database as embeddings for similarity- based retrieval from the independent claims, these limitations do not integrate the judicial exception into a practical application because they add insignificant extra-solution activity to the judicial exception. This limitation merely recites storing and/or retrieving data in a memory, which is extra-solution activity. See MPEP 2106.05(g).
With respect to the limitations for wherein the one or more Al agents are fine-tuned using techniques including data-driven fine-tuning, prompt engineering, parameter optimization, and self-improvement techniques selected from the group consisting of meta-learning, transfer learning, and agent self-analyses, and wherein iteratively optimizing the market research parameters incorporates Reinforcement Learning from Human Feedback (RLHF) based on user feedback from claim 10, and using a chart generator agent and using a code interpreter agent from claim 11, these limitations fail to integrate the abstract idea into a practical application because the provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception.
Accordingly, because the Step 2A Prong One and Prong Two analysis resulted in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
Step 2B: The claims are analyzed to determine whether any additional element, or combination of additional elements, is/are sufficient to ensure that the claims amount to significantly more than the judicial exception. This analysis is also termed a search for "inventive concept." See MPEP 2106.05.
Regarding the computing additional elements, namely a memory unit configured to store machine-readable instructions, a processor operably connected to the memory unit, and a non-transitory computer-readable storage medium from the independent claims, these additional element(s) has/have been evaluated, but fail to add significantly more to the claims because they amount to using generic computing elements (computer hardware) or instructions/software (engine) to perform the abstract idea, similar to adding the words “apply it” (or an equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (network computing environment, the internet, online) and does not amount to significantly more than the abstract idea itself. Applicant’s specification recites the computing additional elements at a high level of generality. Therefore, the additional elements merely describe generic computing elements or computer-executable instructions (software) merely serve to tie the abstract idea to a particular operating environment, which does not add significantly more to the abstract idea. See, e.g., Alice Corp., 134 S. Ct. 2347, 110 USPQ2d 1976; Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
With respect to the limitations for via a user-proxy artificial intelligence (AI) agent executed by a multi-agent framework server of a market research system (MRS), using one or more Al agents within the MRS, the available data sources comprising: (i) non-agent sources including at least one of web scraping, Application Programming Interface (API) endpoints, web browsing, and uploaded files;, using a retrieval augmented generation (RAG) process that retrieves one or more of the embeddings from the vector database responsive to the query, and activating the one or more Al agents within the MRS based on the re-weighted reliability scores, incoming data and feedback, and wherein the multi-agent framework server implements a task planner, and an executor that sequentially implements steps of the plan by coordinating with specialized agents including at least one of a generator, a transformer and a summarizer from the independent claims, these limitations fail to add significantly more to the claims because the provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. See, e.g., Alice Corp., 134 S. Ct. 2347, 110 USPQ2d 1976; Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
With respect to the limitations for receiving a query from a user within the MRS, wherein the query comprises a research objective and at least one of a standard report, and one or more questions, and retrieving data from the available data sources based on the strategy from the independent claims, these limitations fail to add significantly more to the claims because they add insignificant extra-solution activity (e.g., mere data gathering) to the judicial exception. This limitation merely recites receiving and/or sending data over a network, which is extra-solution activity. See MPEP 2106.05(g). Additionally, the mere data gathering extra-solution activity has been recognized as well-understood, routine, and conventional, and thus insufficient to add significantly more to the abstract idea. See MPEP 2106.05(d) - Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
With respect to the limitations for storing at least a portion of the retrieved data in a vector database as embeddings for similarity- based retrieval from the independent claims, these limitations fail to add significantly more to the claims because they add insignificant extra-solution activity to the judicial exception. This limitation merely recites storing and/or retrieving data in a memory, which is extra-solution activity. See MPEP 2106.05(g). Section 2106.05(d)(II) of the MPEP states that “storing and retrieving information in memory” is a well-understood, routine, and conventional computer function. Therefore, this limitation is not anything significantly more than the judicial exception.
With respect to the limitations for wherein the one or more Al agents are fine-tuned using techniques including data-driven fine-tuning, prompt engineering, parameter optimization, and self-improvement techniques selected from the group consisting of meta-learning, transfer learning, and agent self-analyses, and wherein iteratively optimizing the market research parameters incorporates Reinforcement Learning from Human Feedback (RLHF) based on user feedback from claim 10, and using a chart generator agent and using a code interpreter agent from claim 11, these limitations fail to add significantly more to the claims because they provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. See, e.g., Alice Corp., 134 S. Ct. 2347, 110 USPQ2d 1976; Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Accordingly, claims 1-23 are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-9, 12-16, and 18-23 are rejected under 35 U.S.C. 103 as being unpatentable over Cella et al. (US 20230186201 A1, hereinafter “Cella”), in view of Lewis et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, Facebook AI Research; University College London; New York University (2021) (hereinafter “Lewis”), in further view of Rambow et al. (US 12619399 B1, hereinafter “Rambow”), in further view of Balaji et al. (US 20210374776 A1, hereinafter “Balaji”), in further view of Alkurd et al. (US 20210266781 A1, hereinafter “Alkurd”).
Regarding claims 1/9/20: Cella teaches a method, and a computer system, and a non-transitory computer-readable storage medium for automating market research ([0013] A need exists for improved methods and systems for data collection in industrial environments, as well as for improved methods and systems for using collected data to provide improved monitoring, control, intelligent diagnosis of problems and intelligent optimization of operations in various heavy industrial environments.; [1689] The processor, or any machine utilizing one, may include non-transitory memory that stores methods, codes, instructions, and programs as described herein and elsewhere. The processor may access a non-transitory storage medium through an interface that may store methods, codes, and instructions as described herein and elsewhere.) with the following limitations:
collecting context relevant to the query based on availability of internal information retrieval by querying the user with one or more clarification questions using one or more Al agents within the MRS and receiving responses; ([0026] In embodiments, a set of settings for a set of roles may include role-based preference settings. In embodiments, a role-based preference setting may be configured based on a set of role-specific templates. In embodiments, a set of templates may include at least one of a CEO template, a COO template, a CFO template, a counsel template, a board member template, a CTO template, a chief marketing officer template, an information technology manager template, a chief information officer template, a chief data officer template, an investor template, a customer template, a vendor template, a supplier template, an engineering manager template, a project manager template, an operations manager template, a sales manager template, a salesperson template, a service manager template, a maintenance operator template, and a business development template.; [4365] In response, the digital twin simulation system 60320 may return the simulation results to the CMO digital twin 60628, which in turn outputs the results to the user via the client device display. In this way, the user is provided with various outcomes corresponding to different parameter configurations. In some embodiments, the user may select a parameter set based on the various outcomes. In some embodiments, an executive agent trained by the user may select the parameter sets based on the various outcomes.; [4423] In embodiments, a CHRO digital twin 60638 may leverage an executive agent 60704 that is trained on a user's (e.g., an HR executive's) actions (e.g., behaviors, responses, interactions and preferences) using the expert agent system 60010 in response to events and situations encountered by the user (e.g., alerts, notifications, escalations, delegations, presentations of data, events, and the like).);
forming a strategy to retrieve data from available data sources based on the context, wherein the strategy formation comprises determining reliability of the available data sources and prioritizing the available data sources according to relevance to the research objective, ([4366] The CMO digital twin 60628 may utilize the machine learning, A.I. and other analytic capabilities, as described herein, to analyze the content of the four categories of content and classify and score the content characteristics that are probabilistically associated with improved financial or other performance for stated types of marketing campaigns or marketing subject matter.);
the available data sources comprising: (i) non-agent sources including at least one of web scraping, Application Programming Interface (API) endpoints, web browsing, and uploaded files; ([4367] In embodiments, a CMO digital twin 60628 may be configured to store, aggregate, merge, analyze, prepare, report and distribute material relating to market surveys, online surveys, customer panels, ratings, rankings, marketing trend data or other data related to marketing. A CMO digital twin 60628 may link to, interact with, and be associated with external data sources, and able to upload, download, aggregate external data sources, including with the EMP's internal data, and analyze such data, as described herein. Data analysis, machine learning, AI processing, and other analysis may be coordinated between the CMO digital twin 60628 and an analytics team based at least in part on using the artificial intelligence services system 60012. This cooperation and interaction may include assisting with seeding data elements and domains in the enterprise data store 60014 for use in modeling, machine learning, and AI processing to identify the optimal marketing content, sales channels, target consumers, price points, timing, or some other marketing-relating metric or aspect, as well as identification of the optimal data measurement parameters on which to base judgment of a marketing endeavor's success. Examples of data sources 60030 that may be connected to, associated with, and/or accessed from the CMO digital twin 60628 may include, but are not limited to, a sensor system 60032, a sales database 60034 that is updated with sales figures in real time, a CRM system 60038, a marketing campaign platform 60040, news websites, a financial database 60048 that tracks costs of the business, surveys 60050 (e.g., customer satisfaction surveys), an org chart 60052, a workflow management system 60054, customer databases 60062 structured to store customer data, and/or third-party datastores 60060 structured to store third-party data.; [4371] In embodiments, a CMO digital twin 60628 may be configured to monitor, store, aggregate, merge, analyze, prepare, report and distribute material relating to competitors of a CMO's organization, or named entities of interest. In embodiments, such data may be collected by the EMP 60000 via data aggregation, spidering, web-scraping, or other techniques to search and collect competitor information from sources including, but not limited to, press releases, SEC or other financial reports, mergers and acquisitions activity, or some other publicly available data.; [4265] After an enterprise digital twin is served, some enterprise digital twins may be subsequently updated with real-time data received via the API system 60018.);
and (ii) agent-based sources including at least one of synthetic focus groups, synthetic surveys, and autonomous survey collection, ([1754] a machine-learned model may be trained on training data (expert generated data and/or historical data) that corresponds to one or more conditions relating to a particular component.; [4367] In embodiments, a CMO digital twin 60628 may be configured to store, aggregate, merge, analyze, prepare, report and distribute material relating to market surveys, online surveys, customer panels, ratings, rankings, marketing trend data or other data related to marketing.);
retrieving data from the available data sources based on the strategy; ([4371] In embodiments, a CMO digital twin 60628 may be configured to monitor, store, aggregate, merge, analyze, prepare, report and distribute material relating to competitors of a CMO's organization, or named entities of interest. In embodiments, such data may be collected by the EMP 60000 via data aggregation, spidering, web-scraping, or other techniques to search and collect competitor information from sources including, but not limited to, press releases, SEC or other financial reports, mergers and acquisitions activity, or some other publicly available data.);
determining market research parameters based on the analysed data, the market research parameters comprising dynamic reports with visualizations and/or structured reports based on predefined templates; ([4365] the CMO digital twin 60628 may be configured to simulate marketing campaigns, such that the simulations of the marketing campaign may vary parameters such as vehicles (e.g., social media, television, billboards, print, etc.), budget, targeting parameters (e.g., geographic, demographic, or the like), and/or other suitable marketing campaign parameters. In these embodiments, the digital twin simulation system 60320 may receive a request to perform the simulation CMO digital twin, where the request indicates campaign features and the parameters that are to be varied. In response, the digital twin simulation system 60320 may return the simulation results to the CMO digital twin 60628, which in turn outputs the results to the user via the client device display. In this way, the user is provided with various outcomes corresponding to different parameter configurations. In some embodiments, the user may select a parameter set based on the various outcomes. In some embodiments, an executive agent trained by the user may select the parameter sets based on the various outcomes.; [4367] a CMO digital twin 60628 may be configured to store, aggregate, merge, analyze, prepare, report and distribute material relating to market surveys, online surveys, customer panels, ratings, rankings, marketing trend data or other data related to marketing. A CMO digital twin 60628 may link to, interact with, and be associated with external data sources, and able to upload, download, aggregate external data sources, including with the EMP's internal data, and analyze such data, as described herein. Data analysis, machine learning, AI processing, and other analysis may be coordinated between the CMO digital twin 60628 and an analytics team based at least in part on using the artificial intelligence services system 60012. This cooperation and interaction may include assisting with seeding data elements and domains in the enterprise data store 60014 for use in modeling, machine learning, and AI processing to identify the optimal marketing content, sales channels, target consumers, price points, timing, or some other marketing-relating metric or aspect, as well as identification of the optimal data measurement parameters on which to base judgment of a marketing endeavor's success. Examples of data sources 60030 that may be connected to, associated with, and/or accessed from the CMO digital twin 60628 may include, but are not limited to, a sensor system 60032, a sales database 60034 that is updated with sales figures in real time, a CRM system 60038, a marketing campaign platform 60040, news websites, a financial database 60048 that tracks costs of the business, surveys 60050 (e.g., customer satisfaction surveys), an org chart 60052, a workflow management system 60054, customer databases 60062 structured to store customer data, and/or third-party datastores 60060 structured to store third-party data.);
and iteratively optimizing market research parameters by re-weighting reliability scores of the available data sources based on quality metrics derived from the retrieved data, and activating the one or more Al agents within the MRS based on the re-weighted reliability scores, incoming data and feedback; ([0299] Machine learning may be used to improve the foregoing, such as by adjusting one or more weights, structures, rules, or the like (such as changing a function within a model) based on feedback (such as regarding the success of a model in a given situation) or based on iteration (such as in a recursive process). Where sufficient understanding of the underlying structure or behavior of a system is not known, insufficient data is not available, or in other cases where preferred for various reasons, machine learning may also be undertaken in the absence of an underlying model; that is, input sources may be weighted, structured, or the like within a machine learning facility without regard to any a priori understanding of structure, and outcomes (such as those based on measures of success at accomplishing various desired objectives) can be serially fed to the machine learning system to allow it to learn how to achieve the targeted objectives. For example, the system may learn to recognize faults, to recognize patterns, to develop models or functions, to develop rules, to optimize performance, to minimize failure rates, to optimize profits, to optimize resource utilization, to optimize flow (such as flow of traffic), or to optimize many other parameters that may be relevant to successful outcomes (such as outcomes in a wide range of environments). Machine learning may use genetic programming techniques, such as promoting or demoting one or more input sources, structures, data types, objects, weights, nodes, links, or other factors based on feedback (such that successful elements emerge over a series of generations); [0387] By continuously adjusting parameters to cause outputs to match actual conditions, the machine learning facility may self-organize to provide a highly accurate model of the conditions of an environment (such as for predicting faults, optimizing operational parameters, and the like).
Cella doesn’t teach:
receiving, via a user-proxy artificial intelligence (AI) agent executed by a multi-agent framework server of a market research system (MRS), a query from a user within the MRS, wherein the query comprises a research objective and at least one of a standard report, and one or more questions;
wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective; (
and analysing the data to determine if the retrieved data is sufficient to complete the research objective;
storing at least a portion of the retrieved data in a vector database as embeddings for similarity- based retrieval;
analyzing the retrieved data using a retrieval augmented generation (RAG) process that retrieves one or more of the embeddings from the vector database responsive to the query to determine whether the retrieved data is sufficient to complete the research objective;
requesting modification or additional information in the data based on the analysis using the one or more Al agents within the MRS, responsive to the retrieved data not being determined as sufficient to complete the research objective;
wherein the multi-agent framework server implements a task planner that generates a step-by-step plan in response to the query, and an executor that sequentially implements steps of the plan by coordinating with specialized agents including at least one of a generator, a transformer and a summarizer.
Lewis teaches:
and analysing the data to determine if the retrieved data is sufficient to complete the research objective; ([Page 5; 3.4 Fact Verification] The task requires retrieving evidence from Wikipedia relating to the claim and then reasoning over this evidence to classify whether the claim is true, false, or unverifiable from Wikipedia alone. FEVER is a retrieval problem coupled with an challenging entailment reasoning task. It also provides an appropriate testbed for exploring the RAG models’ ability to handle classification rather than generation. We map FEVER class labels (supports, refutes, or not enough info) to single output tokens and directly train with claim-class pairs. Crucially, unlike most other approaches to FEVER, we do not use supervision on retrieved evidence. In many real-world applications, retrieval supervision signals aren’t available, and models that do not require such supervision will be applicable to a wider range of tasks. We explore two variants: the standard 3-way classification task (supports/refutes/not enough info) and the 2-way (supports/refutes) task studied in Thorne and Vlachos [57]. In both cases we report label accuracy.);
storing at least a portion of the retrieved data in a vector database as embeddings for similarity- based retrieval; ([Page 4; 3 Experiments] We use the document encoder to compute an embedding for each document, and build a single MIPS index using FAISS [23] with a Hierarchical Navigable Small World approximation for fast retrieval [37].);
analyzing the retrieved data using a retrieval augmented generation (RAG) process that retrieves one or more of the embeddings from the vector database responsive to the query to determine whether the retrieved data is sufficient to complete the research objective; ([Page 4; 3 Experiments] We experiment with RAG in a wide range of knowledge-intensive tasks. For all experiments, we use a single Wikipedia dump for our non-parametric knowledge source. Following Lee et al. [31] and Karpukhin et al. [26], we use the December 2018 dump. Each Wikipedia article is split into disjoint 100-word chunks, to make a total of 21M documents. We use the document encoder to compute an embedding for each document, and build a single MIPS index using FAISS [23] with a Hierarchical Navigable Small World approximation for fast retrieval [37]. During training, we retrieve the top k documents for each query. We consider k ∈ {5,10} for training and set k for test time using dev data. We now discuss experimental details for each task.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine Cella with Lewis’ feature(s) listed above. One would’ve been motivated to do so in order to classify whether a natural language claim is supported or refuted by Wikipedia, or whether there is not enough information to decide (Lewis; [Page 5]). By incorporating the teachings of Lewis, one would’ve been able to use corrective retrieval augmented generation.
Lewis doesn’t teach:
receiving, via a user-proxy artificial intelligence (AI) agent executed by a multi-agent framework server of a market research system (MRS), a query from a user within the MRS, wherein the query comprises a research objective and at least one of a standard report, and one or more questions;
wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective;
requesting modification or additional information in the data based on the analysis using the one or more Al agents within the MRS, responsive to the retrieved data not being determined as sufficient to complete the research objective;
wherein the multi-agent framework server implements a task planner that generates a step-by-step plan in response to the query, and an executor that sequentially implements steps of the plan by coordinating with specialized agents including at least one of a generator, a transformer and a summarizer.
Rambow teaches:
receiving, via a user-proxy artificial intelligence (AI) agent executed by a multi-agent framework server of a market research system (MRS), a query from a user within the MRS, wherein the query comprises a research objective and at least one of a standard report, and one or more questions; ([Fig. 8] User Prompt: Create a python program that reads JSON input from a REST API with products, prices, unique ID, and quantity. For each product I want a web view that shows the product with all information and allows me to order the product. Each order should decrease the quantity. This is an internal page to manage items in my company. We don’t need payments but we keep the order amount referenced to a project ID that must be provided on checkout.; [Column 18, Lines 28-31] FIG. 8 depicts example operations of an orchestrator agent processing a prompt including a supported task. At operation 802, the orchestrator agent receives a prompt from a client 801.);
wherein the multi-agent framework server implements a task planner that generates a step-by-step plan in response to the query, and an executor that sequentially implements steps of the plan by coordinating with specialized agents including at least one of a generator, a transformer and a summarizer. ([Column 25, Lines 37-48] At operation 1508, the development agent can send a prompt including an action or step from the action plan to a code generation model 1598 (e.g., another generative AI model 197), the prompt including a request to generate code for the indication action or step and, optionally, context for the application (e.g., the application summary). At operation 1510, the development agent can receive the recommended code change(s). As indicated, the development agent can prompt the code generation model 1598 for recommended code changes for each action in the action plan. The development agent can store the recommended code change(s) associated with each action in the action plan.; [Column 26, Lines 1-16] At operation 1508, the development agent can send a prompt including an action or step from the action plan to a code generation model 1598 (e.g., another generative AI model 197), the prompt including a request to generate code for the indication action or step and, optionally, context for the application (e.g., the application summary). At operation 1510, the development agent can receive the recommended code change(s). As indicated, the development agent can prompt the code generation model 1598 for recommended code changes for each action in the action plan. The development agent can store the recommended code change(s) associated with each action in the action plan.; [Column 26, Lines 17-32] The operations 1600 include, at block 1602, receiving a description of a change to a software system. The operations 1600 further include, at block 1604, obtaining data associated with the software system from a data source (see for example the description of block 1208 for further details). The operations 1600 further include, at block 1606, generating, based on the obtained data, a summary of the software system. The operations 1600 further include, at block 1608, prompting a large language model (LLM) to generate an action plan to move the software system from a current state to an updated state that includes the change, the prompt based at least in part on the summary of the software system. The operations 1600 further include, at block 1610, receiving an action plan from the LLM, the action plan including one or more steps to implement the change to the software system.; [Column 11, Lines 53-55] LLMs use a type of neural network called a transformer to process and understand the patterns and structures of language.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Rambow’s feature(s) listed above. One would’ve been motivated to do so in order to leverage a more general-purpose LLM trained on a larger variety of texts (Rambow; [Column 11, Lines 60-61]). By incorporating the teachings of Rambow, one would’ve been able to receive a query from a user including an objective and one or more questions, generate a plan in response to the user query and execute the plan.
Rambow doesn’t teach:
wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective;
requesting modification or additional information in the data based on the analysis using the one or more Al agents within the MRS, responsive to the retrieved data not being determined as sufficient to complete the research objective;
Balaji teaches:
requesting modification or additional information in the data based on the analysis using the one or more Al agents within the MRS, responsive to the retrieved data not being determined as sufficient to complete the research objective; ([0010] Autonomous digital agents in the proposed evaluation system would evaluate the accuracy of collected data arriving from various data collection channels using machine learning models. Further, example autonomous digital agents also use machine learning models to determine whether to obtain additional and/or replacement data when previously collected data is found to be inaccurate and/or otherwise unreliable, thereby enabling the iterative improvement of the data collection accuracy.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Balaji’s feature(s) listed above. One would’ve been motivated to do so in order to evaluate and iteratively improve market research data collection accuracy based on the patterns of various data collection features (or attributes) (Balaji; [0010]). By incorporating the teachings of Balaji, one would’ve been able to determine if more data to complete the research objectives.
Balaji doesn’t teach:
wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective;
Alkurd teaches:
wherein the agent-based sources are generated by simulating one or more user personas representing members of a target demographic for the research objective; ([0189] Data-driven personalization will empower wireless networks to further optimize resources while maintaining user expectations of networks. In order to design, test, and validate research ideas related to wireless network personalization, acquiring data is necessary. However, datasets that comprise user behavior and corresponding user satisfaction information are generally not published due to privacy and confidentiality concerns. To account for this, in this section, we propose a synthetic dataset design methodology to generate labeled user behavior data with ground truth satisfaction values which mimic the real characteristics of real datasets.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Alkurd’s feature(s) listed above. One would’ve been motivated to do so in order to generate labeled user behavior data with ground truth satisfaction values which mimic the real characteristics of real datasets (Alkurd; [0189]). By incorporating the teachings of Alkurd, one would’ve been able to generate synthetic data that mimics members of a target demographic.
Regarding claim 2: Cella teaches:
wherein the internal information retrieval comprises simulating user personas and gathering insights into user preferences and behaviors. ([4247] Executive digital twins may refer to digital twins that are configured for a respective executive within an enterprise. Examples of executive digital twins may include CEO digital twins, CFO (Financial) digital twins, COO (Operations) digital twins, HR digital twins, CTO (Technology) digital twins, CMO (Marketing) digital twins, General Counsel (Legal) digital twins, CIO (Information) digital twins, and the like… an artificial intelligence system may be trained, such as on a labeled industry-specific or domain-specific data set, to automatically generate an industry-specific or domain-specific digital twin for an instance of an EMP for an organization).
Regarding claim 3: Cella doesn’t explicitly teach:
wherein if the internal information retrieval is available, the clarification questions are provided to gather additional details to refine the research objective,
and wherein if the internal information retrieval is not available, the clarification questions are provided to establish a baseline of the context.
Balaji teaches:
wherein if the internal information retrieval is available, the clarification questions are provided to gather additional details to refine the research objective, ([0010] example autonomous digital agents also use machine learning models to determine whether to obtain additional and/or replacement data when previously collected data is found to be inaccurate and/or otherwise unreliable, thereby enabling the iterative improvement of the data collection accuracy.; [0020] Additionally or alternatively, in some examples, particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.; [0023] In some examples, when collection information is determined to be inaccurate (e.g., the discrepancy between predicted and actual values satisfies a threshold), the market research entity 108 may generate a work order or request for new collection information to be obtained. For example, the market research entity 108 may provide instructions to an auditor to return to a particular store 104 associated with the inaccurate collection information and re-collect the relevant information.; [0035] Costs may be affected by the type of data collection channel used to obtain both the initial collection information and any replacement collection information. For instance, sending an auditor into a particular store 104 with a data collector 106 having an auditor application 112 involves more time and expense to the market research entity 108 than requesting collection information from a data collector 106 located at the store 104 with a POS application 110. However, an auditor may be able to provide more accurate and/or complete information than what is available through the POS application 110. Thus, a balance must be struck between the different data collection channels and the associated costs.);
and wherein if the internal information retrieval is not available, the clarification questions are provided to establish a baseline of the context. ([0020] Additionally or alternatively, in some examples, particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.); [0023] Furthermore, as noted above, particular events and/or circumstances may create situations where original collection information and/or replacement collection information is not available. Accordingly, in some examples, the market research entity 108 may generate simulated, synthetic, or synthesized data to replace inaccurate collection information in lieu of obtaining replacement collection information and/or to provide additional collection information when such information is otherwise unavailable for a particular period of interest. In some examples, the synthesized data is generated based on the application of a machine learning model to historical collection information for a particular store 104 of interest and/or for other similar stores 104.; [0035] request the replacement collection information from a different store 104 that is similar and/or otherwise associated with the data packet having data to be replaced (e.g., the different store 104 contains characteristics that would result in the same classification by the characteristics classifier 206).).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Balaji’s feature(s) listed above. One would’ve been motivated to do so in order to enhance market research data collection quality (Balaji; [Abstract]). By incorporating the teachings of Balaji, one would’ve been able to get additional information by asking for clarification.
Regarding claim 4: Cella doesn’t explicitly teach:
further comprising adjusting the strategy formation based on feedback received from the user.
Balaji teaches:
further comprising adjusting the strategy formation based on feedback received from the user. ([0020] particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.); [0038] this final data (e.g., after all iterations through the process) is provided to the example report generator 224 to generate a report. The report may be provided (e.g., transmitted via the communications interface 202) to the product provider(s) 102 and/or the stores(s) 104 to use as appropriate (e.g., adjust marketing campaigns, restock inventory, etc.).).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Balaji’s feature(s) listed above. One would’ve been motivated to do so in order to provide to the data packet database 218 new and/or additional data packets that may be analyzed (Balaji; [0038]). By incorporating the teachings of Balaji, one would’ve been able to adjust the strategy.
Regarding claim 5/16: Cella teaches:
wherein the context relevant to the query comprises information about the user including target demographics, company goals, pricing strategies, and product features. ([4357] In embodiments, the types of data that may populate and/or be utilized by a CMO digital twin 60628 may include, but are not limited to, macroeconomic data; market pricing data; competitive product and pricing data; microeconomic analytic data; forecast data; demand planning data; competitive matrix data; product roadmap; product capability data; consumer; consumer profile data; collaborative filtering data; analytic results of AI and/or machine learning modeling; channel data; demographic data; geographic data; prediction data; recommendation data, or some other type of data relevant to the operations of the CMO and/or marketing department.).
Regarding claim 7/18: Cella doesn’t explicitly teach:
iteratively finetuning the one or more AI agents within the MRS based on the data, user feedback, and predefined optimization criteria. ([0299] Machine learning may be used to improve the foregoing, such as by adjusting one or more weights, structures, rules, or the like (such as changing a function within a model) based on feedback (such as regarding the success of a model in a given situation) or based on iteration (such as in a recursive process).; [0424] Embodiments include training a model to identify preferred sensor sets to diagnose a condition of an industrial environment, where a training set is created by a human user and the model is improved based on feedback from data collected about conditions in an industrial environment.; [0430] As noted above, methods and systems are disclosed herein for training AI models based on industry-specific feedback, such as that reflects a measure of utilization, yield, or impact, where the AI model operates on sensor data from an industrial environment.; [0980] This may include optimizing the coordination using an expert system, such as a rule-based optimization, a model-based optimization, or optimization using machine learning.).
Regarding claim 8/19: Cella teaches:
wherein the structured reports provided to the user are generated based on predefined templates utilizing the one or more AI agents within the MRS to fill in relevant data and analysis. ([0482] In embodiments, an expert analysis module 5100 may generate reports 5102 that may use machine or measurement point specific information from the information store 5040 to analyze the stream data 5050 using a stream data analyzer module 5104 and the local data control application 5062 with the extract/process (“EP”) align module 5068. In embodiments, the expert analysis module 5100 may generate new alarms or ingest alarm settings into an alarms module 5108 that is relevant to the stream data 5050. In embodiments, the stream data analyzer module 5104 may provide a manual or automated mechanism for extracting meaningful information from the stream data 5050 in a variety of plotting and report formats.).
Regarding claim 12: Cella teaches:
further comprising: generating a series of recommended prompts based on the user's market research objective, selected competitors, and other prompts; and allowing the user to select from the recommended prompts to ask the system in a way that results in better results. ([4283] a monitoring agent that monitors the manner by which a user responds to specific requests (e.g., a request from the CEO to populate a report) or notifications. The monitoring agent may report the user's response to such prompts to the EMP 60000.).
Regarding claim 13/21: Cella teaches:
wherein for the internal information retrieval, the processor/ computer-executable instructions is/are further enabled to simulate user personas and gather insights into user preferences and behaviors. ([4247] Executive digital twins may refer to digital twins that are configured for a respective executive within an enterprise. Examples of executive digital twins may include CEO digital twins, CFO (Financial) digital twins, COO (Operations) digital twins, HR digital twins, CTO (Technology) digital twins, CMO (Marketing) digital twins, General Counsel (Legal) digital twins, CIO (Information) digital twins, and the like… an artificial intelligence system may be trained, such as on a labeled industry-specific or domain-specific data set, to automatically generate an industry-specific or domain-specific digital twin for an instance of an EMP for an organization).
Regarding claim 14/23: Cella doesn’t explicitly teach:
wherein if the internal information retrieval is available, the clarification questions are provided to gather additional details to refine the research,
and wherein if the internal information retrieval is not available, the clarification questions are provided to establish a baseline of the context.
Balaji teaches:
wherein if the internal information retrieval is available, the clarification questions are provided to gather additional details to refine the research, ([0010] example autonomous digital agents also use machine learning models to determine whether to obtain additional and/or replacement data when previously collected data is found to be inaccurate and/or otherwise unreliable, thereby enabling the iterative improvement of the data collection accuracy.; [0020] Additionally or alternatively, in some examples, particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.);
and wherein if the internal information retrieval is not available, the clarification questions are provided to establish a baseline of the context. ([0020] Additionally or alternatively, in some examples, particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.); [0023] when collection information is determined to be inaccurate (e.g., the discrepancy between predicted and actual values satisfies a threshold), the market research entity 108 may generate a work order or request for new collection information to be obtained. For example, the market research entity 108 may provide instructions to an auditor to return to a particular store 104 associated with the inaccurate collection information and re-collect the relevant information. Obtaining replacement collection information in this manner can be cost prohibitive. Furthermore, as noted above, particular events and/or circumstances may create situations where original collection information and/or replacement collection information is not available. Accordingly, in some examples, the market research entity 108 may generate simulated, synthetic, or synthesized data to replace inaccurate collection information in lieu of obtaining replacement collection information and/or to provide additional collection information when such information is otherwise unavailable for a particular period of interest. In some examples, the synthesized data is generated based on the application of a machine learning model to historical collection information for a particular store 104 of interest and/or for other similar stores 104.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Balaji’s feature(s) listed above. One would’ve been motivated to do so in order to enhance market research data collection quality (Balaji; [Abstract]). By incorporating the teachings of Balaji, one would’ve been able to get additional information by asking for clarification.
Regarding claim 15/22: Cella doesn’t explicitly teach:
the processor/ computer-executable instructions is/are further enabled to adjust the strategy formation based on feedback received from the user.
Balaji teaches:
the processor/ computer-executable instructions is/are further enabled to adjust the strategy formation based on feedback received from the user. ([0020] particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.); [0038] this final data (e.g., after all iterations through the process) is provided to the example report generator 224 to generate a report. The report may be provided (e.g., transmitted via the communications interface 202) to the product provider(s) 102 and/or the stores(s) 104 to use as appropriate (e.g., adjust marketing campaigns, restock inventory, etc.).;
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Balaji’s feature(s) listed above. One would’ve been motivated to do so in order to provide to the data packet database 218 new and/or additional data packets that may be analyzed (Balaji; [0038]). By incorporating the teachings of Balaji, one would’ve been able to adjust the strategy.
Claims 6, 11 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Cella et al. (US 20230186201 A1, hereinafter “Cella”), in view of Lewis et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, Facebook AI Research; University College London; New York University (2021) (hereinafter “Lewis”), in further view of Rambow et al. (US 12619399 B1, hereinafter “Rambow”), in further view of Balaji et al. (US 20210374776 A1, hereinafter “Balaji”), in further view of Alkurd et al. (US 20210266781 A1, hereinafter “Alkurd”) as applied to claim 1 above, in further view of Froman et al. (US 20200394680 A1, hereinafter “Froman”).
Regarding claim 6/17: Cella doesn’t explicitly teach:
displaying the market research parameters including the dynamic reports with the visualizations including charts, graphs, and tables to convey the analysed data.
Froman teaches:
displaying the market research parameters including the dynamic reports with the visualizations including charts, graphs, and tables to convey the analysed data. (Figs. 5-21 show a series of graphical user interfaces that teach showing users graphs, charts and tables based on the market research performed.; [0202] At 1310, a visualization of the selected results metric is displayed. Examples of visualizations include but are not limited to charts, graphs, tables, and the like. At 1312, The user may be able to toggle between different visualizations of the results data, such as between a pie chart and a bar graph.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Froman’s feature(s) listed above. One would’ve been motivated to do so in order to generate a survey report from the response data, and transmit the survey report to a respondent device (Froman; [0009]). By incorporating the teachings of Froman, one would’ve been able to display visualizations to convey analyzed data.
Regarding claim 11: Cella teaches:
further comprising: extracting information from the data returned by the one or more AI agents using a chart generator agent; ([4253] In embodiments, the EMP trains and deploys expert agents on behalf of enterprise users. In embodiments, an expert agent is an AI-based software agent; [4286] In embodiments, the digital twin configuration system 60302 may configure the databases that support each respective enterprise digital twin of an enterprise (e.g., role-based digital twins, environment digital twins, organizational digital twins, process digital twins, and the like), which may be stored on the digital twin data store 60334. In embodiments, for each database configuration, the digital twin configuration system 60302 may identify and connect any external resources needed to collect data for each respective data type. For example, certain executive digital twins (e.g., CEO digital twin, CFO digital twin, COO digital twin, and CMO digital twin) may each require data derived and/or obtained from a CRM 60038 of the enterprise. In this example, the digital twin configuration system 60302 may configure one or more data collection threads to access an API, SDK, port, search facility, database access facility, and/or other connection facilities of the CRM 60038 of the enterprise on behalf of the enterprise and may obtain any necessary security credentials to access the API. In another example, in order to collect data from one or more edge devices 60064 of the enterprise, the configuration system 60302 may initiate a process of granting access to the edge devices 60064 of the enterprise to the APIs of the EMP 60000.; [4367] In embodiments, a CMO digital twin 60628 may be configured to store, aggregate, merge, analyze, prepare, report and distribute material relating to market surveys, online surveys, customer panels, ratings, rankings, marketing trend data or other data related to marketing.).
Cella doesn’t teach:
selecting a chart template from a series of chart templates or using a code interpreter agent and a data visualization library to generate a custom chart; and filling the extracted information into the selected chart template or custom chart to generate visualizations of the market research parameters.
Froman teaches:
selecting a chart template from a series of chart templates or using a code interpreter agent and a data visualization library to generate a custom chart; and filling the extracted information into the selected chart template or custom chart to generate visualizations of the market research parameters. ([0201] At 1308, the user can select a results metric. Examples of results metrics include but are not limited to overall likability, message intrusion, ad intrusion, confusing, unique, relevant, fits with brand.; [0202] At 1310, a visualization of the selected results metric is displayed. Examples of visualizations include but are not limited to charts, graphs, tables, and the like. At 1312, The user may be able to toggle between different visualizations of the results data, such as between a pie chart and a bar graph.; [0203] In some cases, interface 1301 may include interface 1700. In such cases, the interface 1301 also displays demographic results. The demographic results may include a demographic metric selection and a visualization.).
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Froman’s feature(s) listed above. One would’ve been motivated to do so in order to toggle between visualizations for the results data (Froman; [0214]). By incorporating the teachings of Froman, one would’ve been able to choose between different visualization templates.
Claims 10 is rejected under 35 U.S.C. 103 as being unpatentable over Cella et al. (US 20230186201 A1, hereinafter “Cella”), in view of Lewis et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, Facebook AI Research; University College London; New York University (2021) (hereinafter “Lewis”), in further view of Rambow et al. (US 12619399 B1, hereinafter “Rambow”), in further view of Balaji et al. (US 20210374776 A1, hereinafter “Balaji”), in further view of Alkurd et al. (US 20210266781 A1, hereinafter “Alkurd”) as applied to claim 1 above, in further view of Wei et al. (US 20230244938 A1, hereinafter “Wei”).
Regarding claim 10: Cella teaches:
wherein the one or more AI agents are fine-tuned using techniques including data-driven fine-tuning,… …parameter optimization, ([0299] Machine learning may be used to improve the foregoing, such as by adjusting one or more weights, structures, rules, or the like (such as changing a function within a model) based on feedback (such as regarding the success of a model in a given situation) or based on iteration (such as in a recursive process).; [0424] Embodiments include training a model to identify preferred sensor sets to diagnose a condition of an industrial environment, where a training set is created by a human user and the model is improved based on feedback from data collected about conditions in an industrial environment.; [0430] As noted above, methods and systems are disclosed herein for training AI models based on industry-specific feedback, such as that reflects a measure of utilization, yield, or impact, where the AI model operates on sensor data from an industrial environment.; [0980] This may include optimizing the coordination using an expert system, such as a rule-based optimization, a model-based optimization, or optimization using machine learning.);
and wherein iteratively optimizing the market research parameters incorporates Reinforcement Learning from Human Feedback (RLHF) based on user feedback. ([0010] example autonomous digital agents also use machine learning models to determine whether to obtain additional and/or replacement data when previously collected data is found to be inaccurate and/or otherwise unreliable, thereby enabling the iterative improvement of the data collection accuracy.; [0020] Additionally or alternatively, in some examples, particular individuals (e.g., store managers and/or employees, auditors, etc.) may enter their observations directly onto the data collectors 106 (e.g., via a keyboard and/or touchscreen) as part of the data collection process.; [0299] Machine learning may be used to improve the foregoing, such as by adjusting one or more weights, structures, rules, or the like (such as changing a function within a model) based on feedback (such as regarding the success of a model in a given situation) or based on iteration (such as in a recursive process).);
Cella doesn’t teach:
…prompt engineering,… ([Page 3]
and self-improvement techniques selected from the group consisting of meta-learning, transfer learning, and agent self-analyses,
Wei teaches:
…prompt engineering,… ([0066] Instructive sequence 204 can include an instructive trace 208 documenting intermediate states from the instructive query 206 to the instructive response 210. For instance, although the direct answer to the posed query is captured by the instructive response 210, “The answer is 11,” the instructive trace 208 can capture a series of intermediates (or the “chain of thought”) leading to the ultimate answer. For instance, a first intermediate state can include a declaration of a known: “Roger started with 5 balls.” A second intermediate state can include a statement of multiplication based on the query values: “2 cans of 3 tennis balls each is 6 tennis balls.” A third intermediate state can include a summation step (e.g., optionally numeric, in natural language, etc.): “5+6=11.”);
and self-improvement techniques selected from the group consisting of meta-learning, transfer learning, and agent self-analyses, ([0041] In some embodiments, a chain of thought can span multiple queries processed by the machine-learned model. For instance, a target query may include a complex or multi-part question. The target query can be broken down or reduced into one or more query components (e.g., using prompting or other methods, using the same or a different model, etc.). The query components can then be recursively processed by the model. For instance, a first query component can be processed in view of an initial instructive sequence (e.g., a chain-of-thought prompt as described herein, etc.). In some embodiments, each successive query component can be processed in view of prior query components and responses thereto. For instance, in this manner, the machine-learned model can self-construct an updated instructive sequence with each recursion to leverage its own prior work to build toward an ultimate response to the target query.
It would have been obvious to one of ordinary skill in the art, at the time of applicant’s invention, to combine modified Cella with Wei’s feature(s) listed above. One would’ve been motivated to do so in order to self-corroborate its “rationale” to improve the robustness of model output and improve accuracy of the ultimate answers (Wei; [0040]). By incorporating the teachings of Wei, one would’ve been able to train the ai agents using prompt engineering and self-improvement techniques.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL J TORRES CHANZA whose telephone number is (571)272-3701. The examiner can normally be reached Monday thru Friday 8am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached on (571)270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.J.T./Examiner, Art Unit 3625
/SARA GRACE BROWN/Primary Examiner, Art Unit 3625
Prosecution Insights