DETAILED ACTION
Status of Claims
• The following is an office action in response to the communication filed 07/02/2026.
• Claims 1, 3, 9-11, 13, and 19-20 are currently pending and have been examined.
• Claims 1-20 are currently pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more. The claims recite an abstract idea. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
First, it is determined whether the claims are directed to a statutory category of invention. See MPEP 2106.03(II). In the instant case, claims 1-10 are directed to a method, and claims 11-20 are directed to a machine. Therefore, claims 1-20 are directed to statutory subject matter under Step 1 of the Alice/Mayo test (Step 1: YES).
The claims are then analyzed to determine if the claims are directed to a judicial exception. See MPEP 2106.04. In determining whether the claims are directed to a judicial exception, the claims are analyzed to evaluate whether the claims recite a judicial exception (Prong 1 of Step 2A), as well as analyzed to evaluate whether the claims recite additional elements that integrate the judicial exception into a practical application of the judicial exception (Prong 2 of Step 2A). See MPEP 2106.04.
Taking claim 1 as representative, claim 1 recites at least the following limitations that are believed to recite an abstract idea:
creating a predefined schema, prior to use in order processing, the predefined schema defining all possible orders, all possible options for each of the possible orders, and an output format for representing a user order;
receiving audio representing an order spoken by a user;
transcribing, in real time and locally, the received audio via a model to generate a transcribed text stream continuously as the user speaks;
providing the transcribed text stream and the predefined schema to a model, wherein the predefined schema defines a structure for output generated, processes the transcribed text stream in view of the predefined schema to generate a structured output object representing the user order; and
receiving the structured output object from the model representing a current status of the user order and, as the user continues to speak, continuously updating the current status of the user order based on the received structured output object.
The above limitations recite the concept of updating an order status based on an order spoken by a user. These limitations, under their broadest reasonable interpretation, fall within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas, enumerated in the MPEP, in that they recite commercial or legal interactions such as advertising, marketing, or sales activities or behaviors. Specifically, the determination regarding listing of an item is sales activity because it pertains to commerce and a transaction list. Further, these limitations, under their broadest reasonable interpretation, fall within the “Mental Processes” grouping of abstract ideas, enumerated in the MPEP, in that they recite concepts performed in the human mind, including observations, evaluations, judgments, and opinions. Specifically, the analysis and determination are observations, evaluations, and judgements. Independent claim 11 recites similar limitations as claim 1 and as such, claim 11 falls within the same identified grouping of abstract ideas. Accordingly, under Prong One of Step 2A of the Alice/Mayo test, claims 1 and 11 recite an abstract idea (Step 2A, Prong One: YES).
Under Prong Two of Step 2A of the MPEP, claims 1 and 11 recite additional elements, such as an order terminal; a machine-readable output format; audio signals; an artificial intelligence (AI) transcription model trained; the Al transcription model; a large language model (LLM); a server comprising at least one processor and an non-transitory computer-readable storage medium; and the non-transitory computer-readable storage medium comprising executable instructions which, when executed by at least one processor in the server, cause the at least one processor to perform operations. These additional elements are described at a high level in Applicant’s specification without any meaningful detail about their structure or configuration. As such, these computer-related limitations are not found to be sufficient to integrate the abstract idea into a practical application. Although these additional computer-related elements are recited, claims 1 and 11 merely invoke such additional elements as a tool to perform the abstract idea. Implementing an abstract idea on a generic computer is not indicative of integration into a practical application. Similar to the limitations of Alice, claims 1 and 11 merely recite a commonplace business method (i.e., updating an order status based on an order spoken by a user) being applied on a general purpose computer. See MPEP 2106.05(f). Furthermore, claims 1 and 11 generally link the use of the abstract idea to a particular technological environment or field of use. The courts have identified various examples of limitations as merely indicating a field of use/technological environment in which to apply the abstract idea, such as specifying that the abstract idea of monitoring audit log data relates to transactions or activities that are executed in a computer environment, because this requirement merely limits the claims to the computer field, i.e., to execution on a generic computer (see FairWarning v. Iatric Sys.). Likewise, claims 1 and 11 specifying that the abstract idea of updating an order status based on an order spoken by a user is executed in a computer environment merely indicates a field of use in which to apply the abstract idea because this requirement merely limits the claims to the computer field, i.e., to execution on a generic computer. As such, under Prong Two of Step 2A of the MPEP, when considered both individually and as a whole, the limitations of claims 1 and 11 are not indicative of integration into a practical application (Step 2A, Prong Two: NO).
Since claims 1 and 11 recite an abstract idea and fail to integrate the abstract idea into a practical application, claims 1 and 11 are “directed to” an abstract idea (Step 2A: YES).
Next, under Step 2B, the claims are analyzed to determine if there are additional claim limitations that individually, or as an ordered combination, ensure that the claim amounts to significantly more than the abstract idea. See MPEP 2106.05. The instant claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception for at least the following reasons.
Returning to independent claims 1 and 11, these claims recite additional elements, such as an order terminal; a machine-readable output format; audio signals; an artificial intelligence (AI) transcription model trained; the Al transcription model; a large language model (LLM); a server comprising at least one processor and an non-transitory computer-readable storage medium; and the non-transitory computer-readable storage medium comprising executable instructions which, when executed by at least one processor in the server, cause the at least one processor to perform operations. As discussed above with respect to Prong Two of Step 2A, although additional computer-related elements are recited, the claims merely invoke such additional elements as a tool to perform the abstract idea. See MPEP 2106.05(f). Moreover, the limitations of claims 1 and 11 are manual processes, e.g., receiving information, analyzing information, etc. The courts have indicated that mere automation of manual processes is not sufficient to show an improvement in computer-functionality (see MPEP 2106.05(a)(I)). Furthermore, as discussed above with respect to Prong Two of Step 2A, claims 1 and 11 merely recite the additional elements in order to further define the field of use of the abstract idea, therein attempting to generally link the use of the abstract idea to a particular technological environment, such as the Internet or computing networks (see Ultramercial, Inc. v. Hulu, LLC. (Fed. Cir. 2014); Bilski v. Kappos (2010); MPEP 2106.05(h)). Similar to FairWarning v. Iatric Sys., claims specifying that the abstract idea of updating an order status based on an order spoken by a user is executed in a computer environment merely indicates a field of use in which to apply the abstract idea because this requirement merely limits the claim to the computer field, i.e., to execution on a generic computer.
Even when considered as an ordered combination, the additional elements do not add anything that is not already present when they are considered individually. In Alice Corp., the Court considered the additional elements “as an ordered combination,” and determined that “the computer components…‘[a]dd nothing…that is not already present when the steps are considered separately’ and simply recite intermediated settlement as performed by a generic computer.” Id. (citing Mayo, 566 U.S. at 79, 101 USPQ2d at 1972). Similarly, viewed as a whole, claims 1 and 11 simply convey the abstract idea itself facilitated by generic computing components. Therefore, under Step 2B of the Alice/Mayo test, there are no meaningful limitations in claims 1 and 11 that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself (Step 2B: NO).
Dependent claims 2-10 and 12-20, when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because they recite an abstract idea, are not integrated into a practical application, and do not add “significantly more” to the abstract idea. More specifically, dependent claims 2-10 and 12-20 further fall within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas, enumerated in the MPEP, in that they further recite commercial or legal interactions such as advertising, marketing, or sales activities or behaviors and managing personal behavior or relationships or interactions between people. Further, these claims, under their broadest reasonable interpretation, fall within the “Mental Processes” grouping of abstract ideas, enumerated in the MPEP, in that they recite concepts performed in the human mind, including observations, evaluations, judgments, and opinions. Dependent claims 9-10 and 19-20 fail to identify additional elements and as such, are not indicative of integration into a practical application. Dependent claims 2-8 and 12-18 further identify additional elements, such as a display; a server; a wide area network; a WebSocket interface; OpenAI Whisper; and JSON object format. Similar to discussion above the with respect to Prong Two of Step 2A, although additional computer-related elements are recited, the claims merely invoke such additional elements as a tool to perform the abstract idea. See MPEP 2106.05(f). As such, under Step 2A, dependent claims 2-10 and 12-20 are “directed to” an abstract idea. Similar to the discussion above with respect to claims 1 and 11, dependent claims 2-10 and 12-20 analyzed individually and as an ordered combination, invoke such additional elements as a tool to perform the abstract idea and merely indicate a field of use in which to apply the abstract idea because this requirement merely limits the claims to the computer field, i.e., to execution on a generic computer, and therefore, do not amount to significantly more than the abstract idea itself. See MPEP 2106.05(f)(2). Accordingly, under the Alice/Mayo test, claims 1-20 are ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 8-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over previously cited Dorch et al. (US 20220318860 A1), hereafter Dorch, in view of newly cited Pattabiraman et al. (US 20250078820 A1), hereinafter Pattabiraman.
In regards to claim 1, Dorch discloses a method for real-time voice-based order processing in an order terminal, comprising (Dorch: [0019]; [0030]; [0102-0103]):
creating a predefined schema, prior to use in order processing, the predefined schema defining all possible orders, all possible options for each of the possible orders, and an output format for representing a user order (Dorch: [0055] – “the order may use hierarchical structures, with each particular type of order, such as pizza, wings, taco, or the like, at a highest level and modifiers at a lower level in the hierarchy. For example, pizza may be at the highest level while half-pepperoni, half-sausage, double cheese, and the like may be at a lower level…the order may use a flat system with every possible combination as a unique item. For example, (a) half-pepperoni may be a first item, (b) half-sausage may be a second item, (c) double cheese may be a third item, (d) half-pepperoni and half-sausage may be a fourth item, (e) half-pepperoni, half-sausage, and double cheese may be a fifth item”; [0058] – “the menu 136 of the restaurant 132 of FIG. 1 may be represented as a ontology 250 (e.g., a set of menu items in the menu 136 that shows each menu item's properties and the relationships between menu items). In some aspects, the ontology 250 may be represented in the form of a vector. e.g., each type of pizza may have a corresponding vector representation. In some aspects, the menu representations may be generated from unlabeled data, to enable the NLP pipeline 112 to handle any type of information related to ordering, dishes, and food items”; [0052] – “The encoder 210 may be a pre-trained language model 232…may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.)…Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level. Alternately, the pre-trained language model 232 may use a flat system with every possible combination as a unique item”; [0032] – “the order post 104 or the device 156 may display text of what the customer is ordering, e.g., ‘hash browns and a cup of coffee’ and display contents of the cart 126”):
receiving audio signals representing an order spoken by a user (Dorch: [0048] and Fig. 2 – “The NLP pipeline 112 may receive the utterances 115 of a customer (e.g., from the order post 104 or from the customer device 156 of FIG. 1). The NLP pipeline 112 may process audio data 205 that includes at least a portion of the utterances 115 using a speech-to-text converter 206 to convert the audio data 205 to text 207. For example, the utterances 115 may be ‘I would like 2 large pizzas with peperoni and mushrooms.’”);
transcribing, in real time and locally at the order terminal, the received audio signals via an artificial intelligence (AI) transcription model to generate a transcribed text stream continuously as the user speaks (Dorch: [0048] and Fig. 2 – “The NLP pipeline 112 may receive the utterances 115 of a customer (e.g., from the order post 104 or from the customer device 156 of FIG. 1). The NLP pipeline 112 may process audio data 205 that includes at least a portion of the utterances 115 using a speech-to-text converter 206 to convert the audio data 205 to text 207. For example, the utterances 115 may be ‘I would like 2 large pizzas with peperoni and mushrooms.’”; [0021] – “use an edge appliance that interfaces between one or more order posts and the restaurants' communication system and is capable of executing one or more conversational AI based software agents (referred to herein as ‘software agents’ or ‘chat bots’) to provide software agents to take orders from customers”; [0106] and Fig. 2 – “The edge appliance may…process the audio streams locally, using the speech-to-text 206 of FIG. 2 that may be executing on…the edge appliance 502”; [0102] – “The transcript 536 of the conversation 111 is created substantially in real-time (e.g., less than one second after each word is spoken, each word is transcribed)”; [0017] – “The conversation data is used to train an AI engine to provide a software agent (e.g., sometimes referred to as a “chat bot”)”; see also [0127]);
providing the transcribed text stream and the predefined schema to a large language model (LLM), wherein the predefined schema defines a structure for output generated by the LLM, and wherein the LLM processes the transcribed text stream in view of the predefined schema to generate a structured output object representing the user order (Dorch: [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212. The encoder 210 may be a pre-trained language model 232 that converts the text 207 of the utterances into a vector of numbers. The pre-trained language model 232 may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.). The training is based on the conversation data 136 that has been gathered over time between customers and employees who enter data in the EA-POS 102. The employee entered data may be used as labels for the conversation data 136 when training the various machine learning models described herein. The pre-trained language model 232 associates a specific utterance, e.g., “I want chicken wings”, with a specific action, e.g., entering a chicken wing order into the EA-POS 102. The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item.”); and
receiving the structured output object from the LLM representing a current status of the user order and, as the user continues to speak, continuously updating the current status of the user order based on the received structured output (Dorch: [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212. The encoder 210 may be a pre-trained language model 232 that converts the text 207 of the utterances into a vector of numbers. The pre-trained language model 232 may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.). The training is based on the conversation data 136 that has been gathered over time between customers and employees who enter data in the EA-POS 102. The employee entered data may be used as labels for the conversation data 136 when training the various machine learning models described herein. The pre-trained language model 232 associates a specific utterance, e.g., “I want chicken wings”, with a specific action, e.g., entering a chicken wing order into the EA-POS 102. The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item.”; [0073] – “The encoder 210 creates the cart delta vector 216 that includes corresponding actions to update the cart 126 based on the most recent (e.g., latest turn) of the utterances 115”; [0102] – “the edge appliance 502 may determine changes 538 to the cart 126 during the conversation 111 and store the conversation 111 along with the changes 538 to the cart 126 as the conversation data 136…The transcript 536 of the conversation 111 is created substantially in real-time (e.g., less than one second after each word is spoken, each word is transcribed).).
Dorch further discloses machine-readable output such as JSON (Dorch: [0051]), yet Dorch does not explicitly disclose the schema comprising a machine-readable output format for representing.
However, Pattabiraman teaches a similar food ordering system (Pattabiraman: [0019]), including
the schema comprising a machine-readable output format for representing (Pattabiraman: [0025] – “semantic understander 106 can generate a structured representation of the user request. The structured representation can include structured or unstructured data passed in binary or serialization formats, such as JSON, YAML, XML, CSV, etc. For example, the semantic understander 106 can convert the user request, ‘Can I get a chicken sandwich with honey mustard’, to a JSON format of the user requesting a ‘chicken sandwich’ modified with ‘honey mustard’. The semantic understander 106 can include an instruction-prompted machine learning model, such as a large language model (LLM). The prompt can include a goal of updating instructions based on a current instruction state, conversation history, and utterance. For example, the prompt can include instructing the LLM to be a food ordering agent with a goal of updating an order in JSON based on a current order state, conversation history, and utterance. The order should adhere to a provided schema, where each item in the order should have a separate entry in the ‘items’ array. The output should only be valid JSON.”).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to have included the machine-readable output of Pattabiraman in the method of Dorch because Dorch already discloses structured output and Pattabiraman is merely demonstrating what the output may be. Additionally, it would have been obvious to have included the schema comprising a machine-readable output format for representing as taught by Pattabiraman because JSON is well-known and the use of it in a transcription system would have allowed for fulfillment of requests with high accuracy, regardless of whether there is ambiguity in the user request (Pattabiraman: [0017]).
In regards to claim 2, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the transcribed text stream is provided to a display to provide the user with real time feedback of the spoken order during a current transaction (Dorch: [0032] – “As the customer 128 places an order with one of the software agents 116, the order post 104 or the device 156 may display text of what the customer is ordering, e.g., “hash browns and a cup of coffee” and display contents of the cart 126”).
In regards to claim 3, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the structured output object from the LLM is provided to a display to provide the user with real time feedback of the user order during a current transaction (Dorch: [0032] – “As the customer 128 places an order with one of the software agents 116, the order post 104 or the device 156 may display text of what the customer is ordering, e.g., “hash browns and a cup of coffee” and display contents of the cart 126”).
In regards to claim 4, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the LLM is located on a server located remotely from the order terminal (Dorch: [0100] and Fig. 1 – “The edge appliance 502 may include an interface 503(1) to communicatively couple with the one or more order posts 104 (e.g., using the interface 144 of each order post 104) and an interface 503(2) to communicatively couple with the communication system 404 (e.g., using the interface 406). The edge appliance 502 may include a network interface (I/F) 530 that enables the edge appliance 5022 to connect, via the networks 108, with the servers 106 that are hosting the software agents 116. The edge appliance 502 may include processors 504 and memory 506. In some cases, the edge appliance 502 may be capable of hosting the software agents 116”; [0106] – “edge appliance 502 may enable remote monitoring, remote software updating, and software management from a centralized location that is at a different location from the restaurant 132”; [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212”).
In regards to claim 5, Dorch/Pattabiraman teaches the method of claim 4. Dorch further discloses wherein the order terminal is coupled to the LLM on the remotely located server via a wide area network (Dorch: [0100] and Fig. 1 – “The edge appliance 502 may include an interface 503(1) to communicatively couple with the one or more order posts 104 (e.g., using the interface 144 of each order post 104) and an interface 503(2) to communicatively couple with the communication system 404 (e.g., using the interface 406). The edge appliance 502 may include a network interface (I/F) 530 that enables the edge appliance 5022 to connect, via the networks 108, with the servers 106 that are hosting the software agents 116. The edge appliance 502 may include processors 504 and memory 506. In some cases, the edge appliance 502 may be capable of hosting the software agents 116”; [0106] – “edge appliance 502 may enable remote monitoring, remote software updating, and software management from a centralized location that is at a different location from the restaurant 132”; [0132] – “device 900 may include one or more communication interfaces 906 for exchanging data via the network 110. The communication interfaces 906 can facilitate communications within a wide variety of networks…wireless networks (e.g., WLAN, GSM, CDMA, 802.11, Bluetooth, Wireless USB, ZigBee, cellular, satellite, etc.), the Internet and the like”; [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212”).
In regards to claim 6, Dorch/Pattabiraman teaches the method of claim 5. Dorch further discloses wherein the order terminal communicates with the LLM on the remotely located server via a WebSocket interface (Dorch: [0106] – “The edge appliance 502 may use Web sockets and application programming interfaces (APIs) to control connectivity to the network 108 and the servers 106…edge appliance 502 may enable remote monitoring, remote software updating, and software management from a centralized location that is at a different location from the restaurant 132”; [0132] – “device 900 may include one or more communication interfaces 906 for exchanging data via the network 110. The communication interfaces 906 can facilitate communications within a wide variety of networks…wireless networks (e.g., WLAN, GSM, CDMA, 802.11, Bluetooth, Wireless USB, ZigBee, cellular, satellite, etc.), the Internet and the like”; [0100] and Fig. 1 – “The edge appliance 502 may include an interface 503(1) to communicatively couple with the one or more order posts 104 (e.g., using the interface 144 of each order post 104) and an interface 503(2) to communicatively couple with the communication system 404 (e.g., using the interface 406). The edge appliance 502 may include a network interface (I/F) 530 that enables the edge appliance 5022 to connect, via the networks 108, with the servers 106 that are hosting the software agents 116. The edge appliance 502 may include processors 504 and memory 506. In some cases, the edge appliance 502 may be capable of hosting the software agents 116”; [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212”).
In regards to claim 8, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the output from the LLM is provided in JSON object format (Dorch: [0051] – “Structured data may use a structured language, such as JavaScript Object Notation (JSON)…to represent the data. For example, “two pizzas with sausage and pepperoni” may be represented using structured data as: {“Quantity”: 2, “Item”: “Pizza”, “Modifiers”: [“Pepperoni”, “Sausage” ] }. In structured data 209, each data item has an identifier or some fixed structured meaning and is not subject to natural language meaning or interpretation. The order context 120 captures where the customer and the software agent 116 are in the conversation 111 (e.g., what has already been said), what items are in the cart 126”).
In regards to claim 9, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the order terminal is associated with a restaurant and wherein the predefined schema defines all possible menu items and all possible options for each of the possible menu items (Dorch: [0022] – “an edge appliance includes: (1) an order post interface to connect to one or more order posts, (2) a communication interface to connect to: (i) a communications system of a restaurant, and (ii) a point-of-sale terminal of the restaurant”; [0052] – “The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item”).
In regards to claim 10, Dorch/Pattabiraman teaches the method of claim 1. Dorch further discloses wherein the LLM provides, as part of the output, a message requesting user clarification when the transcribed text stream requests something not included in the generated schema (Dorch: [0108] – “if the software agent 116 is unable to proceed further (e.g., stuck in a loop asking for clarification)”; [0103] – “if the software agent 116 is unable to understand one of the utterances 115 even after asking the customer 128 to repeat the utterance”; Examiner note: The limitation “when the transcribed text stream requests something not included in the generated schema” is a conditional limitation and thus all language dependent on this conditional limitation is granted little to no patentable weight. This applies for the method claims. Nevertheless, the limitation has been fully examined).
In regards to claim 11, Dorch discloses a system for real-time voice-based order processing in an order terminal, comprising (Dorch: [0019]; [0030]; [0102-0103]):
a server comprising at least one processor and a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium comprising a predefined schema file, created prior to use in order processing, the predefined schema file defining all possible orders, all possible options for each of the possible orders, and an output format for representing a user order (Dorch: [0055] – “the order may use hierarchical structures, with each particular type of order, such as pizza, wings, taco, or the like, at a highest level and modifiers at a lower level in the hierarchy. For example, pizza may be at the highest level while half-pepperoni, half-sausage, double cheese, and the like may be at a lower level…the order may use a flat system with every possible combination as a unique item. For example, (a) half-pepperoni may be a first item, (b) half-sausage may be a second item, (c) double cheese may be a third item, (d) half-pepperoni and half-sausage may be a fourth item, (e) half-pepperoni, half-sausage, and double cheese may be a fifth item”; [0058] – “the menu 136 of the restaurant 132 of FIG. 1 may be represented as a ontology 250 (e.g., a set of menu items in the menu 136 that shows each menu item's properties and the relationships between menu items). In some aspects, the ontology 250 may be represented in the form of a vector. e.g., each type of pizza may have a corresponding vector representation. In some aspects, the menu representations may be generated from unlabeled data, to enable the NLP pipeline 112 to handle any type of information related to ordering, dishes, and food items”; [0052] – “The encoder 210 may be a pre-trained language model 232…may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.)…Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level. Alternately, the pre-trained language model 232 may use a flat system with every possible combination as a unique item”; [0032] – “the order post 104 or the device 156 may display text of what the customer is ordering, e.g., ‘hash browns and a cup of coffee’ and display contents of the cart 126”; see also [0123]; [0131]):
the non-transitory computer-readable storage medium comprising executable instructions which, when executed by the at least one processor in the server, cause the at least one processor to perform operations, comprising: receiving audio signals representing an order spoken by a user (Dorch: [0048] and Fig. 2 – “The NLP pipeline 112 may receive the utterances 115 of a customer (e.g., from the order post 104 or from the customer device 156 of FIG. 1). The NLP pipeline 112 may process audio data 205 that includes at least a portion of the utterances 115 using a speech-to-text converter 206 to convert the audio data 205 to text 207. For example, the utterances 115 may be ‘I would like 2 large pizzas with peperoni and mushrooms.’”; see also [0123]; [0131]);
transcribing, in real time and locally at the order terminal, the received audio signals via an artificial intelligence (Al) transcription model to generate a transcribed text stream continuously as the user speaks (Dorch: [0048] and Fig. 2 – “The NLP pipeline 112 may receive the utterances 115 of a customer (e.g., from the order post 104 or from the customer device 156 of FIG. 1). The NLP pipeline 112 may process audio data 205 that includes at least a portion of the utterances 115 using a speech-to-text converter 206 to convert the audio data 205 to text 207. For example, the utterances 115 may be ‘I would like 2 large pizzas with peperoni and mushrooms.’”; [0021] – “use an edge appliance that interfaces between one or more order posts and the restaurants' communication system and is capable of executing one or more conversational AI based software agents (referred to herein as ‘software agents’ or ‘chat bots’) to provide software agents to take orders from customers”; [0106] and Fig. 2 – “The edge appliance may…process the audio streams locally, using the speech-to-text 206 of FIG. 2 that may be executing on…the edge appliance 502”; [0102] – “The transcript 536 of the conversation 111 is created substantially in real-time (e.g., less than one second after each word is spoken, each word is transcribed)”; [0017] – “The conversation data is used to train an AI engine to provide a software agent (e.g., sometimes referred to as a “chat bot”)”; see also [0127]);
providing the transcribed text stream and the predefined schema to a large language model (LLM), wherein the predefined schema file defines a structure for output generated by the LLM, and wherein the LLM processes the transcribed text stream in view of the predefined schema to generate a structured output object representing the user order (Dorch: [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212. The encoder 210 may be a pre-trained language model 232 that converts the text 207 of the utterances into a vector of numbers. The pre-trained language model 232 may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.). The training is based on the conversation data 136 that has been gathered over time between customers and employees who enter data in the EA-POS 102. The employee entered data may be used as labels for the conversation data 136 when training the various machine learning models described herein. The pre-trained language model 232 associates a specific utterance, e.g., “I want chicken wings”, with a specific action, e.g., entering a chicken wing order into the EA-POS 102. The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item.”); and
receiving the structured output object from the LLM representing a current status of the user order and, as the user continues to speak, continuously updating the current status of the user order based on the received structured output (Dorch: [0052] – “An encoder 210 of the NLP pipeline 112 receives the text 207 and the structured data 209 as input and predicts an utterance vector 212. The encoder 210 may include a pre-trained language model 232 that predicts, based on the most recent utterances 115 and the current order context 120, (1) how the cart 126 is to be modified and (2) what the software agent 116 provides as a response, e.g., dialog response 220. The encoder 210 is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector 212. The encoder 210 may be a pre-trained language model 232 that converts the text 207 of the utterances into a vector of numbers. The pre-trained language model 232 may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.). The training is based on the conversation data 136 that has been gathered over time between customers and employees who enter data in the EA-POS 102. The employee entered data may be used as labels for the conversation data 136 when training the various machine learning models described herein. The pre-trained language model 232 associates a specific utterance, e.g., “I want chicken wings”, with a specific action, e.g., entering a chicken wing order into the EA-POS 102. The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item.”; [0073] – “The encoder 210 creates the cart delta vector 216 that includes corresponding actions to update the cart 126 based on the most recent (e.g., latest turn) of the utterances 115”; [0102] – “the edge appliance 502 may determine changes 538 to the cart 126 during the conversation 111 and store the conversation 111 along with the changes 538 to the cart 126 as the conversation data 136…The transcript 536 of the conversation 111 is created substantially in real-time (e.g., less than one second after each word is spoken, each word is transcribed).).
Dorch further discloses machine-readable output such as JSON (Dorch: [0051]), yet Dorch does not explicitly disclose the schema comprising a machine-readable output format for representing.
However, Pattabiraman teaches a similar food ordering system (Pattabiraman: [0019]), including
the schema comprising a machine-readable output format for representing (Pattabiraman: [0025] – “semantic understander 106 can generate a structured representation of the user request. The structured representation can include structured or unstructured data passed in binary or serialization formats, such as JSON, YAML, XML, CSV, etc. For example, the semantic understander 106 can convert the user request, ‘Can I get a chicken sandwich with honey mustard’, to a JSON format of the user requesting a ‘chicken sandwich’ modified with ‘honey mustard’. The semantic understander 106 can include an instruction-prompted machine learning model, such as a large language model (LLM). The prompt can include a goal of updating instructions based on a current instruction state, conversation history, and utterance. For example, the prompt can include instructing the LLM to be a food ordering agent with a goal of updating an order in JSON based on a current order state, conversation history, and utterance. The order should adhere to a provided schema, where each item in the order should have a separate entry in the ‘items’ array. The output should only be valid JSON.”).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to have included the machine-readable output of Pattabiraman in the system of Dorch because Dorch already discloses structured output and Pattabiraman is merely demonstrating what the output may be. Additionally, it would have been obvious to have included the schema comprising a machine-readable output format for representing as taught by Pattabiraman because JSON is well-known and the use of it in a transcription system would have allowed for fulfillment of requests with high accuracy, regardless of whether there is ambiguity in the user request (Pattabiraman: [0017]).
In regards to claims 12-16 and 18, all the limitations in system claims 12-16 and 18 are closely parallel to the limitations of method claims 2-6 and 8 analyzed above and rejected on the same bases.
In regards to claim 19, Dorch/Pattabiraman teaches the system of claim 11. Dorch further discloses wherein the order terminal is associated with a restaurant and wherein the predefined schema file defines all possible menu items and all possible options for each of the possible menu items (Dorch: [0022] – “an edge appliance includes: (1) an order post interface to connect to one or more order posts, (2) a communication interface to connect to: (i) a communications system of a restaurant, and (ii) a point-of-sale terminal of the restaurant”; [0052] – “The pre-trained language model 232 predicts what items from the menu 136 are to be added to the cart 126 (e.g., based on one or more actions associated with the utterance 115) and which items are to be removed from the cart 126, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder 210 may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model 232 may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model 232 may use a flat system with every possible combination as a unique item”).
In regards to claim 20, Dorch/Pattabiraman teaches the system of claim 11. Dorch further discloses wherein the LLM provides, as part of the output, a message requesting user clarification when the transcribed text stream requests something not included in the predefined schema file (Dorch: [0108] – “if the software agent 116 is unable to proceed further (e.g., stuck in a loop asking for clarification)”; [0103] – “if the software agent 116 is unable to understand one of the utterances 115 even after asking the customer 128 to repeat the utterance”; Examiner note: The limitation “when the transcribed text stream requests something not included in the generated schema” is a conditional limitation and thus all language dependent on this conditional limitation is granted little to no patentable weight. This applies for the method claims. Nevertheless, the limitation has been fully examined).
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Dorch, in view of Pattabiraman, in view of previously cited Hirshberg et al. (US 20240370661 A1), hereinafter Hirshberg.
In regards to claim 7, Dorch/Pattabiraman teaches the system of claim 1,
yet Dorch does not explicitly disclose wherein the Al transcription model is OpenAI Whisper.
However, Hirshberg teaches a similar transcription method (Hirshberg: [0044]), including
wherein the Al transcription model is OpenAI Whisper (Hirshberg: [0056] – “Generic AI models can also be used…OpenAI's Whisper”; [0044] – “speech-to-text models trained to convert audio content comprising spoken utterances (i.e., speech) into a transcript of textual representations of the spoken utterances”).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to have included the model of Hirshberg in the method of Dorch because Dorch already discloses models and Hirshberg is merely demonstrating what the model may be. Additionally, it would have been obvious to have included wherein the Al transcription model is OpenAI Whisper as taught by Hirshberg because the model is well-known and the use of it in a transcription system would have facilitated more accurate audio labeling (Hirshberg: [0047]).
In regards to claim 17, all the limitations in system claim 17 are closely parallel to the limitations of method claim 7 analyzed above and rejected on the same bases.
Response to Arguments
Applicant’s arguments, filed 07/02/2026, have been fully considered.
35 U.S.C. § 101
Applicant argues that the claims do not recite an abstract idea because the characterization “oversimplifies the claims…claimed operations cannot be practically performed in the human mind”. Remarks pages 6-7. The examiner disagrees. As shown in the rejection of claims under 35 U.S.C. 101 above, the limitations directed to the abstract idea are directly quoted and concepts within the identified as belonging to the Certain Methods of Organizing Human Activity and Mental Processes groupings of abstract ideas. With respect to the instant claims, an order terminal; a machine-readable output format; audio signals; an artificial intelligence (AI) transcription model trained; the Al transcription model; a large language model (LLM); a server comprising at least one processor and an non-transitory computer-readable storage medium; and the non-transitory computer-readable storage medium comprising executable instructions which, when executed by at least one processor in the server, cause the at least one processor to perform operations have been analyzed as additional elements and accordingly are not analyzed under Step 2A, Prong 1. The claims further recite updating an order status based on an order spoken by a user. These claims fall into the Methods of Organizing Human Activity grouping, which includes activity that falls within the enumerated sub-grouping of in that they recite commercial or legal interactions such as advertising, marketing, or sales activities or behaviors. Specifically, updating a current status of an order based on user speech is a sales activity. Further, these limitations, under their broadest reasonable interpretation, fall within the “Mental Processes” grouping of abstract ideas, enumerated in the MPEP, in that they recite concepts performed in the human mind, including observations, evaluations, judgments, and opinions. It is noted that, contrary to Applicant’s assertion, a human using pen and paper can make a real time transcription of audio, generate structures output objects corresponding to predefined schema, and continuously update during the ongoing order. These claims recite the mental process of collecting information, analyzing it, and displaying certain results of the collection and analysis. Accordingly, these claims recite an abstract idea.
Applicant argues the claims are integrated into a practical application because the claims “recite a specific technological solution…improves system responsiveness and reduces latency” (Remarks page 8). The examiner disagrees. The MPEP provides guidance on how to evaluate whether claims recite an improvement in the functioning of a computer or an improvement to other technology or technical field. For example, the MPEP states “the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement.” The MPEP further states that “[t]he specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art,” and that, “conversely, if the specification explicitly sets forth an improvement but in a conclusory manner…the examiner should not determine the claim improves technology” (see MPEP 2106.04). That is, the claim includes the components or steps of the invention that provide the improvement described in the specification. Looking to the specification is a standard that the courts have employed when analyzing claims as it relates to improvements in technology. For example, in Enfish, the specification provided teaching that the claimed invention achieves benefits over conventional databases, such as increased flexibility, faster search times, and smaller memory requirements. Enfish LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36 (Fed. Cir. 2016). Additionally, in Core Wireless the specification noted deficiencies in prior art interfaces relating to efficient functioning of the computer. Core Wireless Licensing v. LG Elecs. Inc., 880 F.3d 1356 (Fed Cir. 2018). With respect to McRO, the claimed improvement, as confirmed by the originally filed specification, was “…allowing computers to produce ‘accurate and realistic lip synchronization and facial expressions in animated characters…’” and it was “…the incorporation of the claimed rules, not the use of the computer, that “improved [the] existing technological process” by allowing the automation of further tasks”. McRO, Inc. v. Bandai Namco Games America Inc., 837 F.3d 1299, (Fed. Cir. 2016).
While the examiner acknowledges that improvements to the functioning of a computer or to any other technology or technical field may constitute integration into a practical application (see MPEP 2106.05(a)), the instant claims do not provide a technical improvement. Rather, the claims provide an improvement to the abstract idea of updating an order status based on an order spoken by a user. With respect to Applicant’s arguments that the invention improves responsiveness and reduces latency, the examiner notes that the additional elements are merely recited at a high level and are insufficient to show an improvement to technology.
Although the claims include computer technology such as an order terminal; a machine-readable output format; audio signals; an artificial intelligence (AI) transcription model trained; the Al transcription model; a large language model (LLM); a server comprising at least one processor and an non-transitory computer-readable storage medium; and the non-transitory computer-readable storage medium comprising executable instructions which, when executed by at least one processor in the server, cause the at least one processor to perform operations, such elements are merely peripherally incorporated in order to implement the abstract idea. Put another way, these additional elements are merely used to apply the abstract idea of updating an order status based on an order spoken by a user in a technological environment without effectuating any improvement or change to the functioning of the additional elements or other technology. This is unlike the improvements recognized by the courts in cases such as Enfish, Core Wireless, and McRO. Unlike precedential cases, neither the specification nor the claims of the instant invention identify such a specific improvement to computer capabilities. The instant claims are not directed to technological improvements but are directed to improving the business method of updating an order status based on an order spoken by a user. The claimed process, while arguably resulting in a better process for ordering, is not providing any improvement to another technology or technical field as the claimed process is not, for example, improving the server and/or computer components that operate the system. Rather, the claimed process is utilizing data sets related to user orders while still employing the same server and/or computer components used in conventional systems to improve updating an order status based on an order spoken by a user, e.g. a business method, and therefore is merely applying the abstract idea using generic computing components. As such, the claims are not integrated into practical application.
Applicant argues the claims provide significantly more because “an inventive concept may be found in the non-conventional and non-generic arrangement of known elements…a specific combination of elements that is not well-understood, routine, or conventional” (Remarks pages 20-21). The examiner disagrees. The MPEP sets forth that if a claim has been determined to be directed to a judicial exception under revised Step 2A, examiners should then evaluate the additional elements individually and in combination under Step 2B to determine whether they provide an inventive concept (i.e., whether the additional elements amount to significantly more than the exception itself). In this case, Applicant's claims merely recite steps of a method with generic computer components being recited in a generic manner. While electronic devices such as an order terminal; a machine-readable output format; audio signals; an artificial intelligence (AI) transcription model trained; the Al transcription model; a large language model (LLM); a server comprising at least one processor and an non-transitory computer-readable storage medium; and the non-transitory computer-readable storage medium comprising executable instructions which, when executed by at least one processor in the server, cause the at least one processor to perform operations are included within the claims, they are claimed in a generic manner and merely perform generic functions. The additional elements are merely peripherally incorporated in order to implement the abstract idea. Put another way, these additional elements are merely used to apply the abstract idea of updating an order status based on an order spoken by a user in a technological environment without effectuating any improvement or change to the functioning of the additional elements or other technology. Applicant’s disclosure does not articulate or suggest how these additional elements function, individually or in combination, in any manner other than using generic functionality nor does the disclosure articulate how the elements provide a technical improvement. Accordingly, the additional elements do not amount to significantly more because they merely amount to using the additional elements as a tool to perform the abstract idea. Accordingly, the claims do not provide significantly more.
35 U.S.C. § 103
Applicant argues the claims are allowable because the cited art does not disclose “creating a predefined schema, prior to use in order processing, the predefined schema defining all possible orders, all possible options for each of the possible orders, and a machine-readable output format for representing a user order” (Remarks pages 8-12). The examiner disagrees. Initially, the examiner notes that the amendments have necessitated a new grounds of rejection and a new reference has been cited to teach a machine-readable output format for representing. Further, Dorch is cited as disclosing creating a predefined schema, prior to use in order processing, the predefined schema defining all possible orders, all possible options for each of the possible orders, and an output format for representing a user order. Dorch discloses this at least in [0052], which discloses that a pre-trained model may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.) and modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level or the pre-trained language model may use a flat system with every possible combination as a unique item. Thus, this schema used by the model is pre-defined prior to use in order processing and defines every possible combination. This schema is further illustrated in [0055]. Dorch further discloses in [0058] – that the menu of the restaurant may be represented as a ontology (e.g., a set of menu items in the menu that shows each menu item's properties and the relationships between menu items). In some aspects, the ontology may be represented in the form of a vector. e.g., each type of pizza may have a corresponding vector representation. In some aspects, the menu representations may be generated from unlabeled data, to enable the NLP pipeline to handle any type of information related to ordering, dishes, and food items. Thus, a format for representing an order is defined. Accordingly, the cited art teaches this limitation in the claim.
Applicant argues the claims are allowable because the cited art does not disclose “providing the transcribed text stream and the predefined schema to a large language model (LLM), wherein the predefined schema defines a structure for output generated by the LLM, and wherein the LLM processes the transcribed text stream in view of the predefined schema to generate a structured output object representing the user order” (Remarks pages 9-12). The examiner disagrees. Dorch discloses this limitation in this claim in [0052], which discloses an encoder of the NLP pipeline receives the text and the structured data as input and predicts an utterance vector. The encoder 210 may include a pre-trained language model that predicts, based on the most recent utterances and the current order context, (1) how the cart is to be modified and (2) what the software agent provides as a response, e.g., dialog response. The encoder is a type of machine learning model that uses a Bidirectional Encoder Representations from Transformers (BERT), e.g., a transformer-based machine learning technique for natural language processing (NLP), to predict the utterance vector. It is noted that a BERT is an LLM. Dorch further discloses in [0052] that the encoder may be a pre-trained language model that converts the text of the utterances into a vector of numbers. The pre-trained language model may be fine-tuned to a specific domain, e.g., to ordering at a restaurant and that too, at a specific type of restaurant (e.g., pizza, wings, tacos, etc.). The training is based on the conversation data that has been gathered over time between customers and employees who enter data in the EA-POS 102. The employee entered data may be used as labels for the conversation data when training the various machine learning models described herein. The pre-trained language model associates a specific utterance, e.g., “I want chicken wings”, with a specific action, e.g., entering a chicken wing order into the EA-POS 102. The pre-trained language model predicts what items from the menu are to be added to the cart (e.g., based on one or more actions associated with the utterance) and which items are to be removed from the cart, and quantities associated with the items that are to be added and/or removed. In some aspects, the encoder may be implemented as a multi-label classifier. Modifiers may include, for example, half pepperoni, half sausage, double cheese, and the like. In some cases, the pre-trained language model may be structured hierarchically, e.g., with pizza at a high level and modifiers at a lower level…the pre-trained language model may use a flat system with every possible combination as a unique item. It is noted that the training with respect to the domain and the task to generate vectors is providing the predefined schema. It is further noted that providing the vectors in view of the domain is generating the structured output object. Thus, the cited art discloses this limitation in the claim.
Applicant argues the claims are allowable because the cited art does not disclose “receiving the structured output object from the LLM representing a current status of the user order and, as the user continues to speak, continuously updating the current status of the user order based on the received structured output object” (Remarks pages 9-12). The examiner disagrees. Dorch discloses this limitation in this claim in [0073] and [0102], which discloses that the encoder produces the vectors to update the cart based on the most recent (e.g., latest turn) of the utterances, where the transcript including changes to the cart is created substantially in real-time (e.g., less than one second after each word is spoken, each word is transcribed).). Dorch additionally discloses in [0052] that the encoder predicts, based on the most recent utterances and the current order context, (1) how the cart is to be modified and (2) what the software agent provides as a response, e.g., dialog response, where the encoder uses a Bidirectional Encoder Representations from Transformers (BERT). The encoder turns the utterances into vectors. This is the received structured output. Thus, the cited art discloses this limitation in this claim.
Applicant argues the claims are allowable because the office action “does not provide an articulated reasoning with a rational underpinning as to why a person of ordinary skill in the art would have modified Dorch in view of Hirshberg” (Remarks pages 11-12). The examiner disagrees. The examiner notes that the amendments have necessitated a new grounds of rejection and Hirshberg is no longer cited in the rejection of the independent claims. Thus, this argument is moot. Furthermore, it is noted with respect to the dependent claims that cite Hirshberg, a rationale is provided. Specifically, because the OpenAI Whisper model is well-known and the use of it in a transcription system would have facilitated more accurate audio labeling (Hirshberg: [0047]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
NPL Reference U, initially cited in the Office action dated 04/28/2026, teaches an automated fast food ordering system. A voice assistant can take orders. The orders can be passed to staff.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANNA MAE MITROS whose telephone number is (571)272-3969. The examiner can normally be reached Monday-Friday from 9:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marissa Thein can be reached at 571-272-6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNA MAE MITROS/Examiner, Art Unit 3689
