DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s amendments and arguments filed 01/29/2026, with respect to claim(s) 1-20 have been fully considered. Applicant amended claims 1, 4-9, 12-17 and 20.
35 U.S.C 101 rejections of Claims 1-20 have been withdrawn in view of the amended claims filed on 01/29/2026.
Applicant’s arguments filed 01/29/2026, with respect to claim(s) 1-20, under 35 U.S.C. 103 have been fully considered but they are not persuasive. Applicant argued that the combination of Thomas and Sobhy Deraz fails to render obvious, "receiving the response from the LLM service, wherein the response includes the selected columns; generating visualization code based on the response by: determining unique identifiers for corresponding ones of the selected columns in the response from a relational database associated with the table; populating a JSON object with the unique identifiers; and executing the visualization code to render, in the user interface, the visualization based on the JSON object". Examiner respectfully disagrees. Previously cited reference of Thomas and Sobhy Deraz does teach the amended limitations. Sobhy Deraz in para.[0026], [0027],[0088]-[0090], teaches that LLM upon receiving the configured prompt, generates a reply which suggests adding a column which computes a total sales quantity based on year-to-date (YTD) sales data. LLM also generates routines to create charts, tables with associated parameters such as columns identifier “Revenue, Expenses, . . . ”. Thomas teaches in column 3, lines 37-46, LLM’s configuring source code based on the JSON data object and executing source code( visualization code) to generate visualization. Column 5, lines 16-19, column 14, lines 28-37, Fig. 5E illustrates the visualization of the spreadsheet. The data structure which defines the chart is based on JSON data object. Therefore, the amended claims are rejected by the previously cited reference of Thomas and Sobhy Deraz. Please see the rejections below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7,9-11 and 15, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Thomas et al. ( US 12321393 B2), hereinafter referenced as Thomas, in view of Sobhy Deraz et al. (US 20240303441 A1), hereinafter referenced as Sobhy Deraz.
Regarding Claim 1, Thomas teaches a method of operating
receiving, in a user interface of the data analytics application, a natural language input from a user relating to a table of the data in the data analytics application, [wherein the table comprises the data organized according to table columns] ( Thomas: Column 11, lines 36-47, 58-66, Figs. 3, 4, prompt engine 305 receives a natural language input from a user via user interface 307, such as in a task pane or chat interface in user interface 307. The input includes a request or query regarding a chart of workbook data 320 which can include tabular data stored by the application in association with a document or spreadsheet) ;
generating a prompt for a large language model (LLM) service, wherein the prompt includes names of the table columns ( Thomas: Column 5, lines 20-35, column 11, lines 66-67, Fig.3, prompt engine 305 configures a prompt based on the input for submission to LLM 330. The prompt can include column header information),
wherein the prompt tasks the LLM service with selecting columns of the table columns for of respective portions of the data based on the natural language input and the names of the table columns ( Thomas: Column 14, lines 6-27, Figs.5D, 5E, based on the user’s natural language input which indicates that the user wants to see the spreadsheet data visualized as a column chart, prompt engine 305 configures a prompt and in response to the prompt, LLM 330 generates reply) ,
and wherein the prompt tasks the LLM service with generating a response in a JSON format ( Thomas: Column 9, lines 27-34, column 14, lines 28-31, LLM generates response in format such as JSON data object);
populating a JSON object with the unique identifiers ( Thomas: Column 10, lines 26-29, column 13, lines 13-28, Fig. 5B illustrates formatting the output as JSON object array which is a representation of chart data and transmits the reply to prompt engine 305, which process the array to modify the chart);
and executing the visualization code to render, in the user interface, ( Thomas: Column 3, lines 37-46, The application receives the reply from the LLM and configures source code based on the JSON data object. The application submits the source code( visualization code) to an execution queue, and when the source code is executed, the visualization is updated according to the reply. Column 5, lines 16-19, column 14, lines 28-37, Fig. 5E illustrates the visualization of the spreadsheet. The data structure which defines the chart is based on JSON data object).
Thomas while teaching the method of claim 1, fails to explicitly teach the claimed, wherein the table comprises the data organized according to table columns; receiving the response from the LLM service, wherein the response includes the selected columns; generating visualization code based on the response by: determining unique identifiers for corresponding ones of the selected columns in the response from a relational database associated with the table;
However, Sobhy Deraz does teach the claimed, wherein the table comprises the data organized according to table columns ( Sobhy Deraz: Para.[0026], the user may submit a request in the user interface of the application to summarize sales data which is listed in multiple columns of a data table of the user's spreadsheet) ;
receiving the response from the LLM service, wherein the response includes the selected columns ( Sobhy Deraz: Para.[0026],[0027], based on user’s request, the application will configure a prompt which contains the substance of the request along with a relevant portion of the spreadsheet ( columns, rows), which it submits to LLM. The LLM, upon receiving the configured prompt, generates a reply which suggests adding a column which computes a total sales quantity based on year-to-date (YTD) sales data);
generating visualization code based on the response by: determining unique identifiers for corresponding ones of the selected columns in the response from a relational database associated with the table ( Sobhy Deraz: Para.[0088]-[0090],Figs. 6A-6E illustrates operational scenario 600 of an LLM integration in a spreadsheet environment. In response to user’s question 637, LLM generates output 638 which includes routines ( visualization code) for creating pivot charts and tables along with associated parameters such as columns identifiers “Revenue, Expenses, . . . ” ( unique identifiers));
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Sobhy Deraz’s teaching of the integration of spreadsheet environments with LLM services, into the method of LLM integrations for data visualization in spreadsheet environment, taught by Thomas, because, the response of a LLM to a user inquiry can be improved by generating multiple alternative suggestions.( Sobhy Deraz [ Para.[0033],[0034]]).
Claim 9 is an apparatus claim, comprising: one or more computer readable storage media ( Thomas: Column 15, lines 34-39, column 16, lines 1-12, Fig. 7 illustrates a computing device 701, storage system 703 include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data); one or more processors operatively coupled with the one or more computer readable storage media ( Thomas: Column 15, lines 34-42, 65-67, Fig. 7, Processing system 702 is operatively coupled with storage system 703); and program instructions stored on the one or more computer readable storage media ( Thomas: Column 15, lines 43-51, Fig. 7, Processing system 702 loads and executes software 705 from storage system 703. Software 705 includes and implements data visualization process 706,), performing the steps in method claim 1 above and as such, claim 9 is similar in scope and content to claim 1 and therefore, claim 9 is rejected under similar rationale as presented against claim 1 above.
Claim 17 is a computer readable storage media claim, when executed by one or more processors of a computing device (Thomas: Column 15, lines 34-42, column 16, lines 1-12, Fig. 7 illustrates a computing device 701, processing system 702 is operatively coupled with storage system 703, storage system 703 include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data), performing the steps in method claim 1 above and as such, claim 17 is similar in scope and content to claim 1 and therefore, claim 17 is rejected under similar rationale as presented against claim 1 above.
Regarding Claim 2, Thomas in view of Sobhy Deraz teach the method of claim 1. Thomas further teaches, further comprising: generating a second prompt for the LLM service, wherein the second prompt tasks the LLM service with selecting a visualization type from a set of visualization types based on the natural language input and the response and wherein the second prompt further tasks the LLM service with returning an object identifier corresponding the selected visualization type ( Thomas: Column 7, lines 36-67, column 8, lines 25-36, Fig. 1 illustrates a set of visualization ( FY22, FY23). Application service generates a second prompt based on user’s second input, after selecting FY22. LLM service 120 generates a reply to the prompt which indicates a chart property to modify and the value that the chart property should be given. Upon receiving the reply to the second prompt, application service 110 generates source code based on the reply and executes the code to modify the chart, as illustrated in user experience 145. Column 4, lines 37-54, the prompt rule may task the LLM to select a chart property and a new or updated value of the property. As illustrated in Fig. 5D, the prompt rule shows a source code, id: 'Chart_Type', [ name: 'Chart Type', value: 'pie', ... ]}, ... ], where with the “value”, the user can select the chart type);
and populating the JSON object with the object identifier ( Thomas: Column 5, lines 16-18, column 13, lines 24-28, JSON data object includes property identifiers ( as object identifier)).
Claim 10 is an apparatus claim, performing the steps in method claim 2 above and as such, claim 10 is similar in scope and content to claim 2 and therefore, claim 10 is rejected under similar rationale as presented against claim 2 above.
Claim 18 is a computer readable storage media claim, performing the steps in method claim 2 above and as such, claim 18 is similar in scope and content to claim 2 and therefore, claim 18 is rejected under similar rationale as presented against claim 2 above.
Regarding Claim 3, Thomas in view of Sobhy Deraz teach the method of claim 2. Thomas further teaches, further comprising classifying the natural language input as a request to create the visualization and generating the prompt based on a prompt template corresponding to the classification ( Thomas: Column 12, lines 1-14, 58-67, Figs. 5D, 5E, prompt engine 305 identifies a prompt template for configuring a prompt relating to a data visualization such as a chart. By using the selected template, prompt engine 305 configures a prompt to include the user’s natural language input. Prompt engine may task the LLM 330 to identify whether the user is trying to interact with a representation of chart 503 ("Prop_Tree") and a rule to return the reply in a particular output format which is suitable for parsing, such as “ id: 'Chart_Type', [ name: 'Chart Type', value: 'pie', ... ]}, ... ],” to classify the types of visualization ).
Claim 11 is an apparatus claim, performing the steps in method claim 3 above and as such, claim 11 is similar in scope and content to claim 3 and therefore, claim 11 is rejected under similar rationale as presented against claim 3 above.
Claim 19 is a computer readable storage media claim, performing the steps in method claim 3 above and as such, claim 19 is similar in scope and content to claim 3 and therefore, claim 19 is rejected under similar rationale as presented against claim 3 above.
Regarding Claim 7, Thomas in view of Sobhy Deraz teach the method of claim 2. Thomas further teaches, wherein the second prompt further tasks the LLM service with generating a natural language caption for the visualization ( Thomas: Column 3, lines 42-46, column 4, lines 37-43, column 5, lines 12-16, Chart tile may be a property. LLM can be tasked to generate or to format a value of a chart property such as the title ( caption). The application receives the reply from the LLM and configures source code based on the data object. The application submits the source code to an execution queue, and when the source code is executed, the value is generated or changed).
and wherein executing the visualization code to render, in the user interface, the visualization further comprises rendering and displaying the natural language caption in association with the visualization ( Thomas: Column 5, lines 47-62, displaying the natural language titles on the chart interface).
Claim 15 is an apparatus claim, performing the steps in method claim 7 above and as such, claim 15 is similar in scope and content to claim 7 and therefore, claim 15 is rejected under similar rationale as presented against claim 7 above.
Claims 4-6, 12-14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Thomas et al. ( US 12321393 B2), hereinafter referenced as Thomas, in view of Sobhy Deraz et al. (US 20240303441 A1), hereinafter referenced as Sobhy Deraz, further in view of Newman et al. (US 11580127 B1), hereinafter referenced as Newman.
Regarding Claim 4, Thomas in view of Sobhy Deraz teach the method of claim 3. Thomas in view of Sobhy Deraz fail to explicitly teach the claimed, further comprising by the data analytics application: identifying the table columns in the table; generating the unique identifiers for the table columns; and storing the unique identifiers in the relational database prior to generating the visualization code
However, Newman does teach the claimed, further comprising by the data analytics application: identifying the table columns in the table;( Newman: Column 6, lines 59-64, column 10, lines 28-47, Figs. 1, 4, visualization system 102 may identify relational database 118 column identifier ( e.g., "COL_1", "COL_2", in data asset visualization 420)).
generating the unique identifiers for the table columns ( Newman: Column 20, lines 46-64, The data store may include a set of tables with respective sets of columns with associated column identifiers. The column identifiers may be defined in a schema for the database. Column identifiers of the columns in each table may be mapped to various concept objects in an ontology ( unique identifiers));
and storing the unique identifiers in the relational database prior to generating the visualization code ( Newman: Column 6, lines 13-19, column 10, lines 28-47, Figs. 1, 4, data assets inventory knowledge base 116 may store mappings between operation data store 118 and ontology database 120. Data assets inventory knowledge base 116 may also store a common identifier for all column names for a concept. Visualization system 102 may identify relational database 118 column identifier ( e.g., "COL_1", "COL_2", in data asset visualization 420)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Newman’s teaching of user interface for database visualization , into the method of LLM integrations for data visualization in spreadsheet environment, taught by Thomas, because, an improved user interface may be generated that informs a user of the size, categorization, and connections between tables of a database.( Newman [ Column 16, lines 4-13]).
Claim 12 is an apparatus claim, performing the steps in method claim 4 above and as such, claim 12 is similar in scope and content to claim 4 and therefore, claim 12 is rejected under similar rationale as presented against claim 4 above.
Claim 20 is a computer readable storage media claim, performing the steps in method claim 4 above and as such, claim 20 is similar in scope and content to claim 4 and therefore, claim 20 is rejected under similar rationale as presented against claim 4 above.
Regarding Claim 5, Thomas in view of Sobhy Deraz, further in view of Newman teach the method of claim 4, Thomas further teaches, wherein executing the visualization code to render, in the user interface, ( Thomas: Column 3, lines 37-46, The application receives the reply from the LLM and configures source code based on the JSON data object. The application submits the source code( visualization code) to an execution queue, and when the source code is executed, the visualization is updated according to the reply. Column 14, lines 14-37, Fig. 5E illustrates the graphical representation of selected columns).
Claim 13 is an apparatus claim, performing the steps in method claim 5 above and as such, claim 13 is similar in scope and content to claim 5 and therefore, claim 13 is rejected under similar rationale as presented against claim 5 above.
Regarding Claim 6, Thomas in view of Sobhy Deraz, further in view of Newman teach the method of claim 5. Thomas further teaches, further comprising: receiving a second user input relating to the visualization ( Thomas: Column 13, lines 29-30, Fig. 5C, user submits a second input ( same as second user input) relating to chart 503);
classifying the second user input as a request for a revised visualization ( Thomas: Column 14, lines 6-9, Fig. 5D, user wants to visualized the data as column chart ( revised visualization));
submitting a third prompt to the LLM service tasking the LLM service with returning one or more new string values for revising the visualization based on the second user input ( Thomas: Column 14, lines 9-27, Fig. 5D, based on the third input and third prompt to the LLM 330, new chart property value in the source code);
generating [[an]] updated visualization code with ( Thomas: Column 3, lines 37-41, the input may request to change the symbols used on a scatter plot. The LLM replies with a data object, such as JSON data object, which includes the chart property associated with scatter plot symbols and the new property value corresponding to the desired result. Column 14, lines 28-33, generates data object);
and executing updated visualization code, creating the revised visualization based on the updated data object ( Thomas: Column 3, lines 41-46, The application receives the reply from the LLM and configures source code based on the JSON data object. The application submits the source code( visualization code) to an execution queue, and when the source code is executed, the visualization is updated according to the reply. Column 14, lines 32-35, Fig. 5E illustrates revised visualization);
Claim 14 is an apparatus claim, performing the steps in method claim 6 above and as such, claim 14 is similar in scope and content to claim 6 and therefore, claim 14 is rejected under similar rationale as presented against claim 6 above.
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Thomas et al. ( US 12321393 B2), hereinafter referenced as Thomas, in view of Sobhy Deraz et al. (US 20240303441 A1), hereinafter referenced as Sobhy Deraz, further in view of Bouton et al. (US 20210103694 A1), hereinafter referenced as Bouton.
Regarding Claim 8, Thomas in view of Sobhy Deraz teach the method of claim [[7]] 1. Sobhy Deraz further teaches, wherein receiving, in the user interface of the data analytics application, the natural language input from the user comprises, for each of the names of the table columns included in the prompt ( Sobhy Deraz: Para.[0049], the application service receives a natural language input from the user in association with the application. The user keys in a natural language input in a chat interface in the user interface displayed on the computing device. The natural language input may refer to the spreadsheet generally, to a data table of the spreadsheet, to data in the spreadsheet, such as rows or columns of data);
responsive to receiving a portion of the natural language input from the user in a field of the user interface, displaying, in the user interface, a subset of the names of the table columns [corresponding to the portion with which to auto-populate the field of the user interface ]( Sobhy Deraz: Para.[0050], the prompt includes contextual information, such as the chat history and a portion of the spreadsheet including row and column headers and a subset of the data);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Sobhy Deraz’s teaching of the integration of spreadsheet environments with LLM services, into the method of LLM integrations for data visualization in spreadsheet environment, taught by Thomas, because, the response of a LLM to a user inquiry can be improved by generating multiple alternative suggestions.( Sobhy Deraz [ Para.[0033],[0034]]).
Thomas in view of Sobhy Deraz, while teaching the method of claim 8, fail to explicitly teach the claimed, responsive to receiving a portion of the natural language input from the user in a field of the user interface, displaying, in the user interface, a subset of the names of the table columns corresponding to the portion with which to auto-populate the field of the user interface ; receiving a selection of a name among the subset of the names; and auto-populating the field with the name
However, Bouton does teach the claimed, responsive to receiving a portion of the natural language input from the user in a field of the user interface, displaying, in the user interface, a subset of the names of the table columns corresponding to the portion with which to auto-populate the field of the user interface ( Bouton: Para.[0030], [0031], Table 1, Fig. 1 illustrates automatically populating entries in a column. A user interface that enables the use of deep learning based QA capabilities to, instead of answering one question at a time, answer numerous instances of a given questions, where each question is filled with a variable from a proximal column in the spreadsheet. Answers for each of the newly formed questions are automatically retrieved via the NLP software based on the input question) ;
receiving a selection of a name among the subset of the names ( Bouton: Para.[0035], Table 2 illustrates where user can define a template question with variables from multiple alternative columns (variable columns) in the spreadsheet);
and auto-populating the field with the name ( Bouton: Para. [0031], [0035], Fig.1, automatically populating entries in a table );
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Bouton’s teaching of systems and methods for populating electronic documents, and, in particular, automatically filling columns in a spreadsheet, into the method of LLM integrations for data visualization in spreadsheet environment, taught by Thomas in view of Sobhy Deraz, because, the automated addition of data (e.g., rows and/or columns) in a spreadsheet would improve the efficiency of digital information handling. ( Bouton[ Para.[0002]-[0004]]).
Claim 16 is an apparatus claim, performing the steps in method claim 8 above and as such, claim 16 is similar in scope and content to claim 8 and therefore, claim 16 is rejected under similar rationale as presented against claim 8 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NADIRA SULTANA whose telephone number is (571)272-4048. The examiner can normally be reached M-F,7:30 am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D. Shah can be reached on (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NADIRA SULTANA/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
04/23/2026
Prosecution Insights