DETAILED ACTION
This communication is a first Office Action Non-Final rejection on the merits. Claims 1-7 as originally filed are currently pending and are considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The use of the term Android/iOS, which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 1 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of copending Application No. 18/407,448 (reference application).
As per Claim 1. A method for providing food ordering services via a virtual cashier with artificial intelligence within a restaurant, the method comprising:
detecting, by one or more processors, a food ordering request from a user in the restaurant, wherein an artificial intelligence engine is coupled to the one or more processors and a server, wherein the artificial intelligence engine is trained by human experts in the field, wherein the virtual cashier is configured to be displayed in LED/OLED displays, Android/iOS tablets, Laptops/PCs, smartphones, or VR/AR goggles, wherein a set of multi-layer info panels coupled to the one or more processors are configured to overlay graphics on top of the virtual cashier, wherein the virtual cashier is configured to be displayed with an appearance of a real human or a humanoid or a cartoon character, wherein the virtual cashier 's gender, age and ethnicity is determined by the artificial Intelligence's analysis on input from the user, wherein the virtual cashier is configured to be displayed in full body or half body portrait mode, wherein the artificial intelligence engine is configured for real-time speech recognition, speech to text generation, real-time dialog generation, text to speech generation, voice-driven animation, and human avatar generation, wherein the artificial intelligence engine is configured to emulate different voices and use different languages, wherein the one or more processors may put the user in a waiting list if the restaurant is full and will notify the user when the table is ready and will estimate waiting time and communicate with customers; detecting and tracking the user's face, eye, and pose by a set of outward-facing cameras coupled to the one or more processors, wherein a set of touch screens coupled to the one or more processors is configured to allow the user to interact with the virtual cashier by hand; detecting the user's voice by a set of microphones coupled to the one or more processors, wherein the set of microphones are connected to loudspeakers, wherein the set of microphones are enabled to be beamforming, wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual cashier, wherein the virtual cashier is configured to be created based on the appearance of a real human character, a popular cartoon character, wherein the virtual cashier is related to a personality shown in the advertisement of the restaurant;
analyzing the user's profile from audio-visual information gathered by the set of outward- facing cameras and the set of microphones, wherein the user's profile includes the user's audio and facial characteristics;
selecting the user's profile based on matching audio and facial characteristics from a set of profiles in a customer database on the server, wherein the user's profile comprises information on prior food ordering habits, food preferences, and possible food allergies; guiding and suggesting a set of menu items through conversation between the virtual cashier and the user, wherein the conversation is based on the user's profile and analysis of the artificial intelligence engine;
providing options to help make an ordering choice through the conversation; receiving a food ordering input from the user, wherein the food ordering input comprises any food special requests from the user; and transmitting the food ordering input to the kitchen of the restaurant. (Claim 1)
Claim 1 of the instant application fails to disclose methods of payment or financing, and method of delivery of the items; receiving information of orders of any of the items from the user, wherein the information includes information of payment, wherein the user can choose different ways for the payment, wherein the different ways includes usages of credit cards, payment applications, cash, checks, cryptocurrency or other payment means; transmitting the orders to cloud servers of the store; arranging the orders to be delivered to an address, wherein the address is chosen by the user; and adding the orders and payment information to the user’s profile.
Claim 1 of the reference application is directed toward a method for providing sales and customer services via a virtual agents powered with artificial intelligence with an improved method of providing options for payment methods and method of delivery of items.
It would have been obvious to one having skill in the art before the effective filing date, to modify the virtual cashier of the instant application with the virtual agent of the reference application to allow a user to select a payment method and delivery address as taught by the reference application to improve the system in order to provide the user payment and delivery options of a food order.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 4 and 7 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 4 and 7 recite wherein the virtual cashier is configured to be displayed in LED/OLED displays, Android/iOS tablets, Laptops/PCs, smartphones, or VR/AR goggles. The specification does not provide sufficient support as to how the virtual cashier is configured for display on the claimed limitations.
Claims 1, 4 and 7 are replete with limitations that lack sufficient support within the Specifications to satisfy the written description requirement. The specification merely repeat the claim limitations and does not provide enough information to show that the inventor had possession of the claimed invention, essentially a “black box”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 4 and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 4 and 7 contains the trademark/trade name Android/iOS. Where a trademark or trade name is used in a claim as a limitation to identify or describe a particular material or product, the claim does not comply with the requirements of 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 1982). The claim scope is uncertain since the trademark or trade name cannot be used properly to identify any particular material or product. A trademark or trade name is used to identify a source of goods, and not the goods themselves. Thus, a trademark or trade name does not identify or describe the goods associated with the trademark or trade name. In the present case, the trademark/trade name is used to identify/describe Android™ and iOS™ operating systems and, accordingly, the identification/description is indefinite.
Claim 1, 4 and 7 are all method claims that recite: detecting, by one or more processors, a food ordering request from a user in the restaurant…”. The intended scope and the use of the structural limitations on the execution of the “detecting” method renders the scope unclear.
The multiple wherein clauses provide a specific structural limitation in relation to the step of detecting a food ordering request from a user in a restaurant. None of these limitations, however, relate to the detecting and how it may be performed refines the method step of a food ordering request from a user in a restaurant. Instead, it represents a structural limitation disconnected from the preceding step performed.
Moreover, it is unclear how such a limitation — which does not recite a specific step to be performed as part of the method — imparts further limitation on the scope of the method because the limitation set forth a structure and its function rather than a step (e.g., displaying virtual cashiers on at least one of...) that is actively performed.
Similar analysis as discussed above is true for the following limitations:
wherein an artificial intelligence engine is coupled to the one or more processors and a server,
wherein the artificial intelligence engine is trained by human experts in the field,
wherein a set of multi-layer info panels coupled to the one or more processors are configured to overlay graphics on top of the virtual agents,
wherein the virtual agents are configured to be displayed with an appearance of a real human or a humanoid or a cartoon character,
wherein the virtual cashier are configured to be displayed in full body or half body portrait mode,
wherein the artificial intelligence engine is configured for real-time speech recognition, speech to text generation, real-time dialog generation, text to speech generation, voice-driven animation, and human avatar generation,
wherein the artificial intelligence engine is configured to emulate different voices and use different languages, wherein the visual agents are connected via network means;
wherein the one or more processors may put the user in a waiting list if the restaurant is full and will notify the user when the table is ready and will estimate waiting tie and communicate with customers;
With each of the above limitations, it is unclear how each limitation relates back to and further refines what is required by the step of detecting, and unclear how such limitations imparts further limitations on the scope of the method steps performed.
Further regarding claim 1, claim 1 recites wherein the artificial intelligence engine is trained by human experts in the field. It is unclear whether this limitation requires the active step of training the AI engine, or if this is merely a characteristic of the AI engine.
Additionally, the claims recite an “artificial intelligence engine”. The specification provides no definition for an artificial intelligence engine, nor does the specification set forth any examples of an artificial intelligence engine. It is unclear whether this term refers to software (as the term ‘engine’ in the art is often used), an AI ‘model’ (which is what is typically trained), or the combination of an AI model plus software.
Lastly, the artificial intelligence engine is trained by human experts in the field renders the claim indefinite because it is unclear how an engine may be trained, as well as what the field referenced in this limitation may refer to.
Claims 1, 4 and 7 recites wherein the virtual agents’ gender, age and ethnicity is determined by the artificial Intelligence’s analysis on input from the user. The grammatical structure of this claim is ambiguous because it is unclear whether this limitation requires active determination, or if this is merely a characteristic of the agents that has already been determined.
Claims 1, 4 and 7 recites wherein a set of multi-layer info panels coupled to the one or more processors are configured to overlay graphics on top of the virtual agents. The term multi-layer info panel is unclear because the specification provides no further description of the structure of the panel, nor does the specification provide a drawing or figure to allow one of ordinary skill to ascertain the scope of what may or may not comprise the panel.
Claims 1, 4 and 7 recites wherein the set of microphones are enabled to be beamforming. It is unclear whether the microphones are actually beamforming or whether they are only capable of beamforming.
Claims 1, 4 and 7 further recites detecting the user’s voice by a set of beamforming microphones coupled to the one or more processors, wherein the set of microphones are connected to loudspeakers, wherein the set of microphones are enabled to be beamforming, wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual agents, wherein the virtual agents are configured to be created based on the appearance of a real human character. As stated previously, certain limitations do not seem to relate back to or refine what is required by the step of detecting the user’s voice by a set of beamforming microphones coupled to the one or more processors.
More specifically, the wherein clauses wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual agents, and, wherein the virtual agents are configured to be created based on the appearance of a real human character do not appear to have anything to do with the claimed voice detection. They are disconnected from the detecting, and it is further unclear how such clauses — which do not recite a specific step to be performed as part of the method — impart further limitation on the scope of the method.
Claims 1, 4 and 7 further recites detecting and tracking the user’s face, eye, and pose by a set of outward-facing cameras coupled to the one or more processors, wherein a set of touch screens coupled to the one or more processors is configured to allow the user to interact with the virtual cashier by hand. As stated above, these limitations do not seem to relate back to or refine what is required by the step of detecting the user’s voice by a set of beamforming microphones coupled to the one or more processors.
More specifically, the wherein clause wherein a set of touch screens coupled to the one or more processors is configured to allow the user to interact with the virtual cashier by hand, do not appear to have anything to do with the claimed detecting and tracking. They are disconnected from the detecting and tracking, and it is further unclear how such a clause — which does not recite a specific step to be performed as part of the method — imparts further limitation on the scope of the method.
Claims 1, 4 and 7 recites guiding and suggesting a set of items through conversation between the virtual cashier and the user, wherein the conversation is based on the user’s profile and analysis of the artificial intelligence engine.
The wherein clause: wherein the conversation is based on the user’s profile and the analysis of the artificial intelligence engine, renders the claim unclear because it is unclear how the analysis of the artificial intelligence engine is being conducted — which does not recite a specific step to be performed as part of the method — imparts further limitation on the scope of the method.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claim(s) recite(s) a method for providing food ordering services via a virtual cashier with artificial intelligence within a restaurant, the method comprising:
detecting a food ordering request from a user in the restaurant,
detecting and tracking the user's face, eye, and pose
detecting the user's voice by a set of microphones coupled to the one or more processors,
analyzing the user's profile from audio-visual information gathered by the set of outward- facing cameras and the set of microphones, wherein the user's profile includes the user's audio and facial characteristics;
selecting the user's profile based on matching audio and facial characteristics from a set of profiles in a customer database on the server; providing options to help make an ordering choice through the conversation;
guiding and suggesting a set of menu items through conversation between the virtual cashier and the user,
receiving a food ordering input from the user, wherein the food ordering input comprises any food special requests from the user; and
transmitting the food ordering input to the kitchen of the restaurant.
The steps of the method, as drafted, provide a process that, under its broadest reasonable interpretation, covers advertising or marketing or sales activities or a business relationship or commercial interactions such as a customer placing an order at a restaurant, which would include receiving an order from a customer and transmitting an order to a kitchen, for example, placing an order in a restaurant can be completed with a cashier at a register or at a drive through.
If a claim limitation, under its broadest reasonable interpretation, covers advertising or marketing or sales activities or a business relationship, then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application.
In particular, the claim recites an additional element of an artificial intelligence engine is coupled to the one or more processors and a server, wherein the artificial intelligence engine is trained by human experts in the field,
wherein the virtual cashier is configured to be displayed in LED/OLED displays, Android/iOS tablets, Laptops/PCs, smartphones, or VR/AR goggles,
wherein a set of multi-layer info panels coupled to the one or more processors are configured to overlay graphics on top of the virtual cashier, wherein the virtual cashier is configured to be displayed with an appearance of a real human or a humanoid or a cartoon character,
wherein the virtual cashier 's gender, age and ethnicity is determined by the artificial Intelligence's analysis on input from the user, wherein the virtual cashier is configured to be displayed in full body or half body portrait mode,
wherein the artificial intelligence engine is configured for real-time speech recognition, speech to text generation, real-time dialog generation, text to speech generation, voice-driven animation, and human avatar generation, wherein the artificial intelligence engine is configured to emulate different voices and use different languages,
wherein the one or more processors may put the user in a waiting list if the restaurant is full and will notify the user when the table is ready and will estimate waiting time and communicate with customers;
wherein a set of touch screens coupled to the one or more processors is configured to allow the user to interact with the virtual cashier by hand;
wherein the set of microphones are connected to loudspeakers, wherein the set of microphones are enabled to be beamforming, wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual cashier, wherein the virtual cashier is configured to be created based on the appearance of a real human character, a popular cartoon character, wherein the virtual cashier is related to a personality shown in the advertisement of the restaurant;
wherein the user's profile comprises information on prior food ordering habits, food preferences, and possible food allergies;
wherein the conversation is based on the user's profile and analysis of the artificial intelligence engine.
The additional elements alone or in combination are not sufficient to integrate the abstract idea into a practical application. The one or more processors are recited at a high‐level of generality (i.e., as a generic device performing a generic function of executing instructions) such that it amounts to no more than mere instructions to apply the exception using a generic computer component.
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Similarly, an artificial intelligence engine would not be sufficient to amount to significantly more than the judicial
exception. As discussed above with respect to integration of the abstract idea into a practical
application, the additional element of an artificial intelligence engine amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is patent ineligible.
A similar analysis has been applied to claims 4 and 7 which recites essentially the same abstract idea as in claim 1, with the adaptation of calculating a total cost, subtracting a special discount, receiving a payment method for payment and providing a receipt. However, these steps are also considered to be part of the abstract idea as part of a commercial interaction between a cashier and a customer.
The dependent claims also are patent ineligible. For example, claims 2-3 and 5-6 include the step of defining the preferred degree of sweetness of a drink and the food ordering input is incorporated into the user’s profile which further describes the commercial interaction between a customer and cashier and fails to integrate the abstract idea into a practical application.
Allowable Subject Matter
Claims 1-7 are allowable over the prior art. Claims 1-7 remain rejected on other grounds. The Examiner hereby asserts that the totality of the evidence neither anticipates nor renders obvious the particular combination of elements as claimed above. That is, the Examiner emphasizes the claims as a whole and hereby asserts that the totality of the evidence fails to set forth, either explicitly or implicitly, an appropriate rationale for combining or otherwise modifying the available prior art to arrive at the claimed invention. The combination of features, in total, would not have been obvious to one of ordinary skill in the art because any combination of the evidence at hand to reach the combination of features as claimed would require a substantial reconstruction of Applicant’s claimed invention relying on improper hindsight bias. The following is a discussion of exemplary prior art that is relevant to the claimed invention:
Lembersky, U.S. Patent Application Publication 2019/0095775, Lembersky et al., discloses a method for providing food ordering services via a virtual cashier with artificial intelligence within a restaurant, (Figure 1, depicts AI Engine 112, coupled to Processor 116 and Database 124),
wherein the artificial intelligence engine is trained by human experts in the field (pg.5, ¶ [0077] discusses these emotional responses will change over time as the AI learns through multiple user interactions with machine learning to improve the response),
wherein the virtual cashier is configured to be displayed in LED/OLED displays, Android/iOS tablets, Laptops/PCs, smartphones, or VR/AR goggles (pg.5, ¶ [0080] discusses the display may comprise a television (TV), a monitor, a light emitting diode (LED) wall, a projector, liquid crystal displays (LCDs), augmented reality (AR) headset, or virtual reality (VR) headset, light field projection, or any similar or otherwise suitable display),
wherein the virtual cashier is configured to be displayed with an appearance of a real human or a humanoid or a cartoon character (pg.2, ¶ [0022] discusses the AI character or avatar may be based on any associated character model, such as a human, avatar, cartoon, or inanimate object character),
wherein the virtual cashier 's gender, age and ethnicity is determined by the artificial Intelligence's analysis on input from the user (pg.8, ¶ [0102] discusses the device can be configured to modify features of an avatar that is to be presented to the user based on the user(s) themselves, such as how to respond, with what emotion to display, with what words to say, with what tone to speak, with what actions or movements to make, and so on. Further, the device can be configured to select an "avatar type" of the avatar, such as a gender, an age, a real vs. imaginary (e.g., cartoon or fictional character), and so on),
wherein the virtual cashier is configured to be displayed in full body or half body portrait mode (Figure 8, depicts AI Character 865 in full body mode),
wherein the artificial intelligence engine is configured for real-time speech recognition, speech to text generation, real-time dialog generation, text to speech generation, voice-driven animation, and human avatar generation (pg.1, ¶ [0006] discusses convert speech to text, process the text and a response to the text, convert the response back to speech and associated lip-syncing motion, face emotional expression, and/or body animation/position, and then display the response speech through an AI character),
wherein the artificial intelligence engine is configured to emulate different voices and use different languages (pg.2, ¶ [0022] discusses if the user is upset, the character might slouch its shoulders, clasp its hands, and/or respond in a calm voice…pg.4, ¶ [0052] discusses client must upload the phoneme morph targets, and choose an available language package which would translate and understand how to properly use the morph targets to form words, assigning the morph targets to the proper phoneme),
detecting and tracking the user's face, eye, and pose by a set of outward-facing cameras coupled to the one or more processors (Figure 8, Cameras 820),
detecting the user's voice by a set of microphones coupled to the one or more processors (Figure 8, Microphones 830),
wherein the set of microphones are connected to loudspeakers (Figure 8, Speakers 840),
wherein the set of microphones are enabled to be beamforming (Figure 8, Microphone 830),
wherein the virtual cashier is configured to be created based on the appearance of a real human character, a popular cartoon character (pg.6, ¶ [0090] discusses avatar 865 may be, for example, a celebrity, a fictional character, an anthropomorphized object, and so on),
wherein the virtual cashier is related to a personality shown in the advertisement of the restaurant (pg.3, ¶ [0029] discusses Alternatively, if an adult user approaches the system, and the user is correlated to a user that frequents the athletic store in the mall, a sports star character may be used);
analyzing the user's profile from audio-visual information gathered by the set of outward- facing cameras and the set of microphones (pg.3, ¶[0038] discusses the AI character system 100 can store or categorize emotions of characters and/or avatars into selectable groups that can be selected based on the determined mood of a user (indicated by audio and/or visual input of the user),
wherein the user's profile comprises information on prior food ordering habits, food preferences, and possible food allergies (pg.3, ¶ [0032] discusses the system may learn about the clothing size John wears, and the calories he consumed while eating at the food court for any restaurants. The virtual assistant may then suggest lighter food if the user John set in his preferences to help him to watch after his diet);
guiding and suggesting a set of menu items through conversation between the virtual cashier and the user (pg.3, ¶ [0032] discusses the virtual assistant could suggest salad place to eat based on John's information and give an excited look and encouraged tone to stay on the diet),
wherein the conversation is based on the user's profile and analysis of the artificial intelligence engine (pg.3, ¶ [0032] discusses user preferences and ¶ [0033] discusses machine learning algorithms may process the data (from the particular user especially, but also from all users generally), and generate appropriate user responses on the short time period bases).
Mahaffy et al., U.S. Patent Application Publication 2003/0018531. Mahaffey et al. teaches a method for providing food ordering services via a virtual cashier with artificial intelligence within a restaurant (Figure 1, Virtual Cashier 38…pg.6, ¶ [0052] discusses transaction processing system is optimized for use within a fixed location, such as the order processing space of a fast food restaurant…pg.6, ¶ [0050] discusses the customer then interacts at 206 with the CIT 12 in real-time, verbally ordering food. The AI routines in the computer process the interaction at 208 to parse and identify the elements the food order.)
detecting, a food ordering request from a user in the restaurant (pg.4, ¶ [032] discusses the virtual cashier preferably displays images of a face of cashier and auditorily greets, engages, and prompts at 100 the customer to verbally provide a fast food order to the virtual cashier (e.g., "Hello. Please place your order with me."). The customer's verbal orders are received by the microphone 22 of the CIT and transmitted to the computer system 14 where they are processed),
wherein the virtual cashier is related to a personality shown in the advertisement of the restaurant (Mahaffy et al. pg.3, ¶ [0032] discusses the images of the virtual cashier are preferably computer generated and, as such, the face and other features of the cashier may be human-like, animal-like, or whimsical in nature, and may even be representative of a mascot of the restaurant, e.g., Ronald McDonald)
guiding and suggesting a set of menu items through conversation between the virtual cashier and the user (pg.5, ¶ [0046] discusses even after the customer indicates that his or her order is complete, the virtual cashier preferably asks whether the customer would like French fries with an order which does not already include French fries, or whether for a nominal additional sum the customer would prefer to upsize the French fries and drink order, e.g., from medium to large),
providing options to help make an ordering choice through the conversation (pg.5, ¶ [0046] discusses the CIT 12 may prompt the customer via computer-generated voice or displayed text to add other items to the menu list, and a complete order may require multiple interactions between the customer and virtual cashier 38; i.e., after the customer orders a sandwich, if the customer does not on his or her own add additional menu items within a predetermined time period, e.g. two second, the virtual cashier engages the customer and asks whether the customer would like a soft drink and, if so, which size);
receiving a food ordering input from the user (pg.4, ¶ [0032] discusses the customer's verbal orders are received by the microphone 22 of the CIT and transmitted to the computer system 14),
wherein the food ordering input comprises any food special requests from the user; and
transmitting the food ordering input to the kitchen of the restaurant (pg.6, ¶ [0048] discusses the order is sent at 128 to order fulfillment employees (kitchen staff and order assembly personnel) who prepare the order).
Lopez et al., U.S. Patent Application Publication 2018/0096440. Lopez discloses wherein the one or more processors may put the user in a waiting list if the restaurant is full and will notify the user when the table is ready and will estimate waiting time and communicate with customers (Figure 1, depicts Reservations for a restaurant for a selected time).;
Carpenter et al. U.S. Patent 10,628,635. Carpenter et al. Col.6, lines 35-39 teaches a user recognition component, an input device component, an interpreter component, a machine-readable text component, a text-to-audio component, a text-to-visual display generator component, and an output processor component.
wherein a set of multi-layer info panels coupled to the one or more processors are configured to overlay graphics on top of the virtual cashier (Col.10, lines 7-10 discusses The transparent rigid panel 103 may also have at least a portion with touchscreen capability that is capable of receiving user input.),
wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual cashier (Col.10, lines 41-45 discusses the camera 105 then takes at least one, but preferably multiple pictures of the detected user. The one or more pictures are then sent over a communication link to the one or more computing devices 101 to determine if the detected user is a previous user…)
wherein a set of touch screens coupled to the one or more processors is configured to allow the user to interact with the virtual cashier by hand (Col.11, lines 36-40 discusses The input could be received from the user's verbal command through a microphone 106 or through a user's tactile input on transparent rigid panel 103 with touchscreen capabilities);
wherein the user's profile includes the user's audio and facial characteristics (Col.5, lines 64-65 discusses a voice record for the user can be used using the on-board microphone…Col.6, lines 53-56 discusses the system uses hardware for tracking faces in order to assign a name and create a record for the user that is interacting with the system);
selecting the user's profile based on matching audio and facial characteristics from a set of profiles in a customer database on the server (Col.6, lines 56-60 discusses This provides numerous advantages to the software, including the ability to tell the difference between multiple speakers in a group, remember previous users in order to recall preferences, expand upon the current knowledge of a user, and for security),
Col.6, lines 55-60 discusses the ability to tell the difference between multiple speakers in a group, remember previous users in order to recall preferences, expand upon the current knowledge of a
user, and for security.
Navaratnam, U.S. Patent 10,152,719. Navaratnam Col.2, lines 32-35 discusses wherein pictures or voices of the user are configured to be uploaded and processed either on a cloud server or in local or personal devices to analyze and create the virtual cashier (The Digital Actor is created from photographs and short video recordings previously taken from a real presenter…Col.4, lines 10-15 discusses user can create their own video to be integrated into their presentation or introductory video. The output video can be obtained by also adding further modifications such as replacing background, changing size, position and rotation of the Digital Actor, give animations and transitions etc.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kaplan U.S. Patent Application Publication 2023/0230293 discusses a method and apparatus to generate and update virtual personification using artificial intelligence comprising a system configured to perform the following. Receive data associated with a person such as text files, audio files, image files, and video files. Render a virtual personification of the person and output the virtual personification to a user, such as on a display screen. Then, receiving and interpreting a user input to generate a user request, and then updating the virtual personification. The update may include generating an audio output using the text files and the audio files of the person and/or generating a video output using the image files and the video files of the person. The audio output and the video output is presented to the user by the virtual personification and it has not previously occurred by the person or thing represented by the virtual personification. Abstract
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHFORD S HAYLES whose telephone number is (571)270-5106. The examiner can normally be reached M-F 6AM-4PM with Flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fahd Obeid can be reached at 5712703324. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASHFORD S HAYLES/ Primary Examiner, Art Unit 3627