DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the application filed on 11/01/2023 and the amendments and response filed 8/28/2025, and the amendments, remarks, and request for continued examination filed 5/5/2026.
Claims 1, 2, 9-12, and 16-18 are currently amended.
Claims 1-20 have been previously amended.
No claims have been added.
No claims have been cancelled.
Claims 1-20 are currently pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/5/2026 has been entered.
Information Disclosure Statement
The information disclosure statement(s) (IDS(s)) submitted on 2/15/2024, 5/6/2026, and 8/5/2026 have been received and considered.
Response to Amendment
Applicant’s amendments to the Claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed 11/21/2025.
Response to Arguments
Applicant’s arguments, see pages 11-16, filed 5/5/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 USC 103 have been fully considered but are not persuasive, as explained below. Therefore, the rejection has been maintained in substance while being updated for clarity and as necessitated by amendment.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Here, the applicant argues regarding Claim 1 that neither Yamazaki (WO 2021148903) nor Ingebretsen (US 20230344537) teach the entirety of “generat[ing] one or more text prompts comprising one or more natural language instructions to generate natural language output responsive to the computed alertness level of the operator,” where the elements are taught in the prior art by a combination of Yamazaki and Ingebretsen as demonstrated below. The applicant also argues regarding Claim 2 that none of Yamazaki, Ingebretsen, Shinkai (US 10224060), or Sawai (US 20230186653) teach the entirety of “compris[es] a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output and a representation of a subsequent computed alertness level,” where the elements are taught by the prior art by a combination of Yamazaki, Ingebretsen, Shinkai, and Sawai as demonstrated below.
The argument that Yamazaki does not teach the amended “generat[ing] one or more text prompts comprising one or more natural language instructions to generate natural language output responsive to the computed alertness level of the operator,” is not persuasive. Yamazaki Pg 6 ¶ 5 “Step S102 is a step in which the conversation information generation unit generates conversation information 93 using […] biometric information, and preference information possessed by the classifier. From the biological information, it is preferable that conversation information 93 corresponding to a warning or a warning is generated. As an example, in "○○", conversation information 93 regarding health can be generated using biological information. Further, in "○○", conversation information 93 can be generated by combining the temperature in the vehicle and the biological information by using the driving information […] The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply,” demonstrates clearly that the biological data is being used as an input i.e. a prompt and not merely a condition of the conversation information, and further Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” demonstrates clearly that the prompt of the biological information is also being used as an output in natural language form that is responsive to the biological information i.e. the computed alertness level, thus teaching the claim within the broadest reasonable interpretation of “generat[ing] one or more […] prompts comprising one or more […] instructions to generate natural language output responsive to the computed alertness level of the operator.” However, Yamazaki lacks detail regarding the prompt specifically being text in a natural language form.
The remainder of the contested portion of the claim “text [prompts comprising…] natural language [instructions…]” is taught by Ingebretsen within the established context of Yamazaki. Ingebretsen ¶ 0220 as cited teaches the inputting of a specifically natural language text prompt, providing a specific form to the otherwise generally stated inputs into Yamazaki’s conversation information generation unit.
Regarding the argument against Claim 2, Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information” teaches “provid[ing] one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output […] as a subsequent input into the one or more language models,” within the broadest reasonable interpretation of the claimed element. With the context established in the rejection of Claim 1 as described above, Yamazaki converts the driver’s response into linguistic data, previously established in combination with Ingebretsen to constitute a text prompt, and thus teaching “subsequent text prompt comprising a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output,” and further updating the preference information, previously established in Yamazaki Pg 6 ¶ 5 as an input to generate conversation information, teaching that this linguistic data is “provid[ed] […] as a subsequent input into the one or more language models.” However, while Yamazaki thus teaches a subsequent input into the conversation information generation unit, it is silent as to using this input to generate a subsequent response using a subsequent alertness level. Shinkai Col 4 lines 44-67 as cited in the previous and following rejection teaches the specific subsequent nature of the “generation of the second natural language output representing a generated response to the operator response,” within the broadest reasonable interpretation of the element, and Sawai ¶ 0109-0110 as cited in the previous and following rejection teaches the subsequent nature of “a representation of a subsequent computed alertness level computed subsequent to the operator response,” within the broadest reasonable interpretation of the element.
In response to applicant's argument regarding the language models of Yamazaki and Ingebretsen relating to Claim 1, a recitation of the intended use of the claimed invention must result in a structural difference between the claimed invention and the prior art in order to patentably distinguish the claimed invention from the prior art. If the prior art structure is capable of performing the intended use, then it meets the claim. Yamazaki Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 and Pg 6 ¶ 5 as cited above demonstrates both input and output of the “neuro-linguistic programming” of Yamazaki Pg 10 ¶ 1, and Ingebretsen ¶ 0220 as cited above demonstrates a natural language input into the “statistical language models” of Ingebretsen ¶ 0072. Ingebretsen ¶ 0232 further describes “In some examples, natural language processing module 732, dialogue flow processing module 734, and task flow processing module 736 are used collectively and iteratively to infer and define the user's intent, obtain information to further clarify and refine the user intent, and finally generate a response (i.e., an output to the user, […]) to fulfill the user's intent. The generated response is a dialogue response to the speech input that at least partially fulfills the user's intent. Further, in some examples, the generated response is output as a speech output,” clearly also showing analogous natural language output rather than “simply select[ing] which function to call. Regarding the citations of the language models themselves, both Yamazaki Pg 10 ¶ 1 and Ingebretsen ¶ 0072 teach the claimed “one or more language models” within the broadest reasonable interpretation according to the present specification ¶ 0064 lines 5-9 “In some embodiments, a "language model" is a set of statistical or probabilistic functions that (e.g., collectively) performs Natural Language Processing (NLP) in order to understand, learn, and/or generate human natural language content,” with Ingebretsen referring more explicitly to natural language.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 3, 8, 9, 11, 15-16, 18, and 20 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki et al (WO 2021148903, hereinafter “Yamazaki,” all excerpts and citations taken from the attached machine translation) in view of Ingebretsen et al (US 20230344537, hereinafter “Ingebretsen”).
Regarding Claim 1, Yamazaki describes:
One or more processors comprising: (Yamazaki Pg 10 ¶ 4 line 1 “The control unit 21 includes a processor 60,”)
one or more processing units to: receive image data (Yamazaki Pg 4 ¶ 2 line 1 “The calculation unit can perform image analysis,”)
representing one or more portions of an operator of an ego- machine, (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,”)
the image data generated using one or more sensors (Yamazaki Pg 4 ¶ 1 line 1 “The image pickup device has a first image pickup device and a second image pickup device.,”)
of an ego-machine traversing an environment; (Yamazaki Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
based at least on the image data, generate a representation of a computed alertness level of the operator; […] (Yamazaki Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination of a user’s level of alertness that includes “drowsiness”)
generate one or more […] prompts comprising one or more […] instructions to generate a natural language output responsive to the computed alertness level of the operator; (Yamazaki Pg 6 ¶ 5 “Step S102 is a step in which the conversation information generation unit generates conversation information 93 using […] biometric information, and preference information possessed by the classifier. From the biological information, it is preferable that conversation information 93 corresponding to a warning or a warning is generated. As an example, in "○○", conversation information 93 regarding health can be generated using biological information. Further, in "○○", conversation information 93 can be generated by combining the temperature in the vehicle and the biological information by using the driving information […] The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply,” emphasis added, demonstrates clearly that the biological data is being used as an input i.e. a prompt and not merely a condition of the conversation information, and further Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” emphasis added, demonstrates clearly that the prompt of the biological information is also being used as an output in natural language form that is responsive to the biological information i.e. the computed alertness level)
provide the one or more […] prompts as an input (Yamazaki Pg 5 ¶ 6 lines 1-4 “In addition, the biosensor can detect the biometric information of the driver. The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93,”)
into one or more […] models (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching the input of the biosensor data into one of several machine learning models accessible to the device, interpretated according to the present specification ¶ 0064 lines 5-9)
to generate the natural language output responsive to the computed alertness level of the operator; (Yamazaki Pg 4 ¶ 5 lines 1-2 “The conversation information generation unit can generate the first conversation information based on the biological information” the conversational information being analogous to natural language characters, and further demonstrating a natural language output as example by Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” emphasis added.)
and cause presentation, using a display device or sound device associated with the operator of the ego-machine, of a representation of the natural language output. (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
Yamazaki does not teach:
[…] text […]
[…] natural language […]
[…] text […]
[…] language […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…one or more] text [prompts comprising one or more] natural language [instructions…provide the one or more] text [prompts […] into one or more language models […] (Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model interpretated according to the present specification ¶ 0064 lines 5-9))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit to generate conversation information of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 3, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further describes:
wherein the one or more processing units are further to generate the representation of the computed alertness level of the operator based at least on mapping the computed alertness level to a natural language description of the computed alertness level, and including the natural language description of the computed alertness level of the operator in the one or more text prompts. (Yamazaki Pg 3 ¶ 7 line 1 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker,” demonstrating the generation of a natural language description of the alertness (“sleepy”) generated from a result of the biosensor detection of the level of alertness including “drowsiness” as previously cited in Pg 5 ¶ 8 lines 1-6, demonstrating by example a mapping of the sensor result to a natural language descriptor as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Regarding Claim 8, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further describes:
wherein the one or more processors is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; (Yamazaki Pg 4 ¶ 7 “Therefore, one aspect of the present invention can be referred to as an information processing system or an automatic driving support system using the above-mentioned information processing device.”)
a perception system for an autonomous or semi-autonomous machine; (Yamazaki Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
or a system implemented using a robot.
Regarding Claim 9, Yamazaki describes:
A system comprising one or more processing units (Yamazaki Pg 10 ¶ 4 line 1 “The control unit 21 includes a processor 60,”)
to: generate one or more […] prompts comprising one or more […] instructions to generate natural language output responsive to (Yamazaki Pg 6 ¶ 5 “Step S102 is a step in which the conversation information generation unit generates conversation information 93 using […] biometric information, and preference information possessed by the classifier. From the biological information, it is preferable that conversation information 93 corresponding to a warning or a warning is generated. As an example, in "○○", conversation information 93 regarding health can be generated using biological information. Further, in "○○", conversation information 93 can be generated by combining the temperature in the vehicle and the biological information by using the driving information […] The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply,” emphasis added, demonstrates clearly that the biological data is being used as an input i.e. a prompt and not merely a condition of the conversation information, and further Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” emphasis added, demonstrates clearly that the prompt of the biological information is also being used as an output in natural language form that is responsive to the biological information i.e. the computed alertness level)
a computed alertness level of an operator of an ego-machine; (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,” Pg 4 ¶ 5 lines 1-2 “[…] based on the biological information […]” and Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination that the user’s alertness level includes “drowsiness” and Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,” emphasis added)
and cause presentation, using a device associated with the operator of the ego-machine, (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
of a representation of the natural language output responsive to the computed alertness level (Yamazaki Pg 4 ¶ 5 line 1 “The conversation information generation unit can generate the first conversation information […]” teaching generation of the previously established output)
and are generated based at least on providing the one or more […] prompts as an input into one or more […] models hosted on or accessible to the ego-machine. (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching conversation generation by several machine learning models accessible to the device based on the biosensor data)
Yamazaki does not teach:
[…]text[…]
[…]natural language[…]
[…]text[…]
[…]language[…]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…one or more] text [prompts comprising one or more] natural language [instructions…providing the one or more] text [prompts as an input into one or more] language [models…] (Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model interpretated according to the present specification ¶ 0064 lines 5-9))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit to generate conversation information of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 11, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further describes:
wherein the one or more processing units are further to generate the representation of the computed alertness level of the operator based at least on mapping the computed alertness level to a natural language description of the computed alertness level, and include the natural language description of the computed alertness level of the operator in the one or more text prompts. (Yamazaki Pg 3 ¶ 7 line 1 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker,” demonstrating the generation of a natural language description of the alertness (“sleepy”) generated from a result of the biosensor detection of the level of alertness including “drowsiness” as previously cited in Pg 5 ¶ 8 lines 1-6, demonstrating by example a mapping of the sensor result to a natural language descriptor as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Regarding Claim 15, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further describes:
wherein the one or more processors is comprised in at least one of:
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; (Yamazaki Pg 7 ¶ 3 “Step S103 is a step of displaying the object 92 on the object 91. Augmented reality or mixed reality can be realized by superimposing the object 92 on the image visually recognized by the eyeglass function of the wearable device 10. Therefore, the object 92 as shown in FIG. 1D can be displayed via the wearable device 10.”)
a system for generating synthetic data; or
a system for generating synthetic data using AI.
Regarding Claim 16, Yamazaki describes:
A method comprising: receiving a representation of a computed alertness level (Yamazaki Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination of a user’s level of alertness that includes “drowsiness”)
of an operator of an ego-machine; (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,” and Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
generate one or more […] prompts comprising one or more […] instructions to generate natural language output responsive to the computed alertness level of the operator; (Yamazaki Pg 6 ¶ 5 “Step S102 is a step in which the conversation information generation unit generates conversation information 93 using […] biometric information, and preference information possessed by the classifier. From the biological information, it is preferable that conversation information 93 corresponding to a warning or a warning is generated. As an example, in "○○", conversation information 93 regarding health can be generated using biological information. Further, in "○○", conversation information 93 can be generated by combining the temperature in the vehicle and the biological information by using the driving information […] The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply,” emphasis added, demonstrates clearly that the biological data is being used as an input i.e. a prompt and not merely a condition of the conversation information, and further Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” emphasis added, demonstrates clearly that the prompt of the biological information is also being used as an output in natural language form that is responsive to the biological information i.e. the computed alertness level)
providing the one or more […] prompts as an input (Yamazaki Pg 5 ¶ 6 lines 1-4 “In addition, the biosensor can detect the biometric information of the driver. The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93,”)
into one or more […] models (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching the input of the biosensor data into one of several machine learning models accessible to the device, interpretated according to the present specification ¶ 0064 lines 5-9)
to generate the natural language output responsive to the computed alertness level of the operator; (Yamazaki Pg 4 ¶ 5 lines 1-2 “The conversation information generation unit can generate the first conversation information based on the biological information” the conversational information being analogous to natural language characters, and further demonstrating a natural language output as example by Pg 5 ¶ 9 – Pg 6 ¶ 1 line 4 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker. This corresponds to a warning or warning to the driver based on the biosensor. Further, the conversation information generation unit generates conversation information about "○○" extracted from preference information in order to stimulate the driver's brain as conversation information 93,” emphasis added.)
and causing presentation, using a device associated with the operator of the ego-machine, of a representation of the natural language output. (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
Yamazaki does not teach:
[…]text[…]
[…]natural language […]
[…]text[…]
[…]language[…]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…one or more] text [prompts comprising one or more] natural language [instructions…provide the one or more] text [prompts […] into one or more language models […] (Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model interpretated according to the present specification ¶ 0064 lines 5-9))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit to generate conversation information of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 18, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further describes:
wherein the presentation, at the device associated with the operator of the ego-machine, of the representation of the natural language output generated by the one or more language models comprises presentation, using a sound device, (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
of a first phrase utterance that initiates a conversation with the operator (Yamazaki Pg 8 ¶ 2 lines 8-10 “The conversation information generation unit preferably generates question-type conversation information 93a for which the driver needs a reply,”)
based at least on one or more interests of the operator. (Yamazaki Pg 8 ¶ 2 lines 1-6 “Step S112 is a step in which the conversation information generation unit generates conversation information 93a using topic information, biological information, and preference information possessed by the classifier. The classifier preferably extracts information from the topical information that is likely to activate the driver's brain. It is preferable that the conversation information 93a corresponding to the alert or warning is generated from the biometric information. As an example, the conversation information 93a is generated using information that is likely to activate the driver's brain using topical information,” the use of both topic information and preference information likely to activate the driver’s brain here teaching that the conversation is based on interests specific to the operator, further supported by Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information.”)
Regarding Claim 20, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further describes:
wherein the method is performed by at least one of: a system for performing deep learning operations; (Yamazaki Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. […] Deep learning using a neural network, and the like can be used for conversation generation,”)
a system for performing real-time streaming;
a system implemented using an edge device;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources. (Yamazaki Pg 3 ¶ 7 line 1 “The conversation information generation unit has a classifier that learns the user's preference information,” Pg 3 ¶ 7 line 4 “by using the classifier on the cloud,)
Claim(s) 2, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view of Ingebretsen and further in view of Shinkai et al (US 10224060, hereinafter “Shinkai”) and Sawai (US 20230186653, hereinafter “Sawai”)
Regarding Claim 2, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to provide one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output […] as a subsequent input into the one or more language models […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information” teaching converting the driver’s response into linguistic data, previously established in combination with Ingebretsen to constitute a text prompt)
Yamazaki does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
[…] to generate second natural language output representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language output, as a natural language response)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Regarding Claim 10, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to provide one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information” teaching converting the driver’s response into linguistic data, previously established in combination with Ingebretsen to constitute a text prompt)
Yamazaki does not teach:
[…]and a representation of a subsequent computed alertness level computed subsequent to the operator response as a subsequent input into the one or more language models […]
[…] to generate second natural language output representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language output, as a natural language response)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Regarding Claim 17, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further teaches:
further comprising providing one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the natural language output […] as a subsequent input into the one or more language models […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information” teaching converting the driver’s response into linguistic data, previously established in combination with Ingebretsen to constitute a text prompt)
Yamazaki does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
[…] to generate second natural language output representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language output, as a natural language response)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Claim(s) 4, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view Ingebretsen and further in view of Shinkai.
Regarding Claim 4, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein one or more […] text prompts provided as […] input to the one or more language models further include at least one of: (Yamazaki Pg 5 ¶ 6 line 1-6 “The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93. The preference information may be selected from a classification having a large number of registered items, or may be selected from a classification having a small number of registered items,” describing preference information being used as a prompt, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
personalized information of the operator, (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information.”)
a natural language instruction to start a conversation with the operator that aligns with the operator's interests, (Yamazaki Pg 6 ¶ 5 line 8-12 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information. The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply.”)
a zero-shot, one-shot or few shot examples of representative inputs or outputs, (Yamazaki Pg 7 ¶ 8 “In step S011, Internet news and the like acquired by the vehicle control unit via satellite or wireless communication can be collected as topic information. In addition, the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally.”)
or an instruction to send a control signal to cause a particular action of the ego-machine. (Yamazaki Pg 6 ¶ 4 line 6-9 “Therefore, by giving the driver information detected by the wearable device 10 to the vehicle, it is possible to suppress the occurrence of accidents due to looking away driving, dozing driving, and the like. Semi-automatic driving or automatic driving can be canceled based on the driver information. The driver information is also given to the conversation information generation unit.”)
Yamazaki does not teach:
[…] subsequent […] subsequent […]
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] wherein one or more subsequent text prompts provided as subsequent input […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Regarding Claim 12, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further teaches:
wherein one or more […] text prompts provided as […] input to the one or more language models include at least one of, (Yamazaki Pg 5 ¶ 6 line 1-6 “The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93. The preference information may be selected from a classification having a large number of registered items, or may be selected from a classification having a small number of registered items,” describing preference information being used as a prompt, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
personalized information of the operator, (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information.”)
a natural language instruction to start a conversation with the operator that aligns with the operator's interests, (Yamazaki Pg 6 ¶ 5 line 8-12 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information. The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply.”)
a zero-shot, one-shot or few shot examples of representative inputs or outputs, (Yamazaki Pg 7 ¶ 8 “In step S011, Internet news and the like acquired by the vehicle control unit via satellite or wireless communication can be collected as topic information. In addition, the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally.”)
or an instruction to send a control signal to cause a particular action of the ego-machine. (Yamazaki Pg 6 ¶ 4 line 6-9 “Therefore, by giving the driver information detected by the wearable device 10 to the vehicle, it is possible to suppress the occurrence of accidents due to looking away driving, dozing driving, and the like. Semi-automatic driving or automatic driving can be canceled based on the driver information. The driver information is also given to the conversation information generation unit.”)
Yamazaki does not teach:
[…] subsequent […] subsequent […]
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] wherein one or more subsequent text prompts provided provided as subsequent input […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Regarding Claim 19, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further teaches:
wherein one or more […] text prompts provided as […] input to the one or more language models further include at least one of: (Yamazaki Pg 5 ¶ 6 line 1-6 “The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93. The preference information may be selected from a classification having a large number of registered items, or may be selected from a classification having a small number of registered items,” describing preference information being used as a prompt, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
the one or more interests of the operator, (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information.”)
a natural language instruction to start a conversation with the operator that aligns with the one or more interests, (Yamazaki Pg 6 ¶ 5 line 8-12 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information. The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply.”)
a zero-shot, one-shot or few shot examples of representative inputs or outputs, (Yamazaki Pg 7 ¶ 8 “In step S011, Internet news and the like acquired by the vehicle control unit via satellite or wireless communication can be collected as topic information. In addition, the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally.”)
or an instruction to send a control signal to cause a particular action of the ego-machine. (Yamazaki Pg 6 ¶ 4 line 6-9 “Therefore, by giving the driver information detected by the wearable device 10 to the vehicle, it is possible to suppress the occurrence of accidents due to looking away driving, dozing driving, and the like. Semi-automatic driving or automatic driving can be canceled based on the driver information. The driver information is also given to the conversation information generation unit.”)
Yamazaki does not teach:
[…] subsequent […] subsequent […]
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] wherein one or more subsequent text prompts provided provided as subsequent input […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Claim(s) 5 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view of Ingebretsen and further in view of Shinkai, Webster et al (US 20200150934, hereinafter “Webster”) and Wang et al (US 11767024, hereinafter “Wang”).
Regarding Claim 5, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to: […] a location of the ego-machine in the environment; […] (Yamazaki Pg 10 ¶ 5 line 2-5 “The second communication device 62 can obtain the surrounding conditions of the vehicle 20, […] it is possible to obtain information such as speed information and position information of vehicles in the vicinity of the vehicle 20,” teaching determination of the vehicle location relative to the surrounding environment)
[…] determine at least one of: […] traffic data, or event data associated with the geo-location; (Yamazaki Pg 10 ¶ 5 line 1-3 “The second communication device 62 can communicate with the satellite 30 or the wireless communication antenna 31. The second communication device 62 can obtain the surrounding conditions of the vehicle 20, traffic information, current affairs information that can be searched via the Internet, and the like,” and Pg 7 ¶ 8 lines 1-5 “the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally,” current affairs information and topical information both being analogous to event data)
and provide one or more […] text prompts comprising the at least one of: the weather data, the road condition data, the traffic data, the event data, or the geo-location indicator as […] input into the one or more language models; and cause presentation, using the display device or sound device associated with the operator or occupant of the ego-machine, of a natural language phrase representing […] the at least one of: the weather data, the road condition data, the traffic data, or the event data. (Yamazaki Pg 13 ¶ 6 line 1-4 “Further, when the display unit 501 displays map information, traffic information […] etc., the object 510 is displayed on one or more of the display panels 507a, 507b, 507c, and 507d. The information processing device outputs conversation information related to map information, traffic information,” as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki does not teach:
[…] receive a geo-location indicator that represents […]
[…] based at least on the geo-location indicator, […]
[…] weather data,
road condition data, […]
[…] second […] second […]
[…] a summary of […]
Within the same field of endeavor as Yamazaki, Wang teaches:
[…] receive a geo-location indicator that represents […] (Wang Col 5 lines 4-13 “With regard to the sensing of the road environment, the apparatus can sense the road environment by at least one of the following modes: sensing by at least one sensor borne on the apparatus; sensing by at least sensor borne on the vehicle; acquiring information from at least one of the following devices in a communication manner: […] acquiring information by a Global Positioning System (GPS).”)
[…] based at least on the geo-location indicator, […] (Wang Col 19 lines 50-57 “In this embodiment of the present invention, when there is a high-accuracy road map, the apparatus can align, based on the sensed road environment and by using the visual localization information and/or GPS localization information, the current road with the road map, so that the apparatus can acquire road surface information by the road map to generate corresponding target driving assistance information.,”)
[…] road condition data, […] (Wang Col 11 lines 33-37 “The target driving assistance information can comprise, but not be limited to, prompt information related to the traffic safety, driving environment, information indication of traffic and road condition or more, traffic regulations, in-vehicle information, or more.”)
Yamazaki and Wang are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the surroundings-based traffic information used for conversation generation of Yamazaki with the simple addition of Wang’s GPS location and surrounding environment information used to prompt information related to traffic safety, driving environment, and road condition to generate driving assistance information. This modification would be made with a reasonable expectation of success as motivated by increasing driver safety and reduce travelling incident occurrence (Wang Col 1 lines 36-39.)
The combination of Yamazaki and Wang does not teach:
[…] weather data, […]
[…] second […] second […]
[…] a summary of […]
Within the same field of endeavor as Yamazaki and Wang, Webster teaches:
[…] weather data, […] a summary of […] (Webster ¶ 0046 lines 5-14 “the output content 208 represents the output of the voice assistant device 108 […] In one or more implementations, the output content 208 is configured as data for audible output by the voice assistant device 108, e.g., audible speech. Additionally or alternately, the output content 208 may be configured as data for display by the voice assistant device 108 or another device. In response to a voice command such as “what's the weather,” for instance, the output content 208 may be a graphical user interface having a summary of upcoming weather and configured for output via a display of the voice assistant device,” teaching both weather data and summarization of data to be presented)
Yamazaki, Wang, and Webster are all considered analogous because they all relate to assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the surroundings-based traffic information used for conversation generation of Yamazaki and Wang’s GPS location and surrounding environment information used to prompt information related to traffic safety, driving environment, and road condition to generate driving assistance information with the simple addition of Webster’s weather information and summarization of data to be presented. This modification would be made with a reasonable expectation of success as motivated by improving conciseness and usefulness of data presented to the operator.
The combination of Yamazaki, Wang, and Webster does not teach:
[…] second […] second […]
Within the same field of endeavor as Yamazaki, Wang, and Webster, Shinkai teaches:
[…] provide one or more second […] prompts […] as second input into the one or more language models […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki, Wang, Webster, and Shinkai are all considered analogous because they all relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Regarding Claim 13, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to: […] a location of the ego-machine; […] (Yamazaki Pg 10 ¶ 5 line 2-5 “The second communication device 62 can obtain the surrounding conditions of the vehicle 20, […] it is possible to obtain information such as speed information and position information of vehicles in the vicinity of the vehicle 20,” teaching determination of the vehicle location relative to the surrounding environment)
[…] determine at least one of: […]
[…] traffic data,
or event data associated with the geo-location; (Yamazaki Pg 10 ¶ 5 line 1-3 “The second communication device 62 can communicate with the satellite 30 or the wireless communication antenna 31. The second communication device 62 can obtain the surrounding conditions of the vehicle 20, traffic information, current affairs information that can be searched via the Internet, and the like,” and Pg 7 ¶ 8 lines 1-5 “the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally,” current affairs information and topical information both being analogous to event data)
and provide one or more […] text prompts comprising the at least one of: the weather data, the road condition data, the traffic data, the event data, or the geo-location indicator as […] input into the one or more language models; and cause presentation, using the device associated with the operator of the ego-machine, of the representation of a natural language phrase representing the at least one of: the weather data, the road condition data, the traffic data, or the event data. (Yamazaki Pg 13 ¶ 6 line 1-4 “Further, when the display unit 501 displays map information, traffic information […] etc., the object 510 is displayed on one or more of the display panels 507a, 507b, 507c, and 507d. The information processing device outputs conversation information related to map information, traffic information,” as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki does not teach:
[…] receive a geo-location indicator that represents […]
[…] based at least on the geo-location indicator, […]
[…]weather data,
road condition data, […]
[…] second […] second […]
Within the same field of endeavor as Yamazaki, Wang teaches:
[…] receive a geo-location indicator that represents […] (Wang Col 5 lines 4-13 “With regard to the sensing of the road environment, the apparatus can sense the road environment by at least one of the following modes: sensing by at least one sensor borne on the apparatus; sensing by at least sensor borne on the vehicle; acquiring information from at least one of the following devices in a communication manner: […] acquiring information by a Global Positioning System (GPS).”)
[…] based at least on the geo-location indicator, […] (Wang Col 19 lines 50-57 “In this embodiment of the present invention, when there is a high-accuracy road map, the apparatus can align, based on the sensed road environment and by using the visual localization information and/or GPS localization information, the current road with the road map, so that the apparatus can acquire road surface information by the road map to generate corresponding target driving assistance information.,”)
[…] road condition data, […] (Wang Col 11 lines 33-37 “The target driving assistance information can comprise, but not be limited to, prompt information related to the traffic safety, driving environment, information indication of traffic and road condition or more, traffic regulations, in-vehicle information, or more.”)
Yamazaki and Wang are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the surroundings-based traffic information used for conversation generation of Yamazaki with the simple addition of Wang’s GPS location and surrounding environment information used to prompt information related to traffic safety, driving environment, and road condition to generate driving assistance information. This modification would be made with a reasonable expectation of success as motivated by increasing driver safety and reduce travelling incident occurence (Wang Col 1 lines 36-39.)
The combination of Yamazaki and Wang does not teach:
[…]weather data, […]
[…] second […] second […]
Within the same field of endeavor as Yamazaki and Wang, Webster teaches:
[…] weather data, […] (Webster ¶ 0046 lines 5-14 “the output content 208 represents the output of the voice assistant device 108 […] In one or more implementations, the output content 208 is configured as data for audible output by the voice assistant device 108, e.g., audible speech. Additionally or alternately, the output content 208 may be configured as data for display by the voice assistant device 108 or another device. In response to a voice command such as “what's the weather,” for instance, the output content 208 may be a graphical user interface having a summary of upcoming weather and configured for output via a display of the voice assistant device,” teaching both weather data and summarization of data to be presented)
Yamazaki, Wang, and Webster are all considered analogous because they all relate to assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the surroundings-based traffic information used for conversation generation of Yamazaki and Wang’s GPS location and surrounding environment information used to prompt information related to traffic safety, driving environment, and road condition to generate driving assistance information with the simple addition of Webster’s weather information. This modification would be made with a reasonable expectation of success as motivated by improving usefulness of data presented to the operator.
The combination of Yamazaki, Wang, and Webster does not teach:
[…] second […] second […]
Within the same field of endeavor as Yamazaki, Wang, and Webster, Shinkai teaches:
[…] provide one or more second […] prompts […] as second input into the one or more language models […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki, Wang, Webster, and Shinkai are all considered analogous because they all relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Claim(s) 6, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view of Ingebretsen and further in view of Webster and Shinkai.
Regarding Claim 6, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to: […] provide one or more […] text prompts comprising a representation of the audio data as […] input into the one or more language models; (Yamazaki Pg 3 ¶ 7 lines 4-8 “by using the classifier on the cloud, the usage history of the information processing device used by the user […] for example, a DVD playback title, a TV program watched). It is possible to make the classifier learn the history of contents, […] as preference information,” and Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, […] using preference information.”) teaching the use of media content, and specifically music (audio data) as preference information input into the machine learning model, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
and cause presentation, using the display device or sound device associated with the operator or occupant of the ego-machine, […] of the representation of the audio data. (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, […] using preference information.”)
Yamazaki does not teach:
[…] based at least on a geo-location indicator, access first audio data
associated with a radio station,
wherein the geo-location indicator represents a location of the ego-machine in the environment; and […]
[…] second […] second […]
[…] of a summary […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…] based at least on a geo-location indicator, (Ingebretsen ¶ 0244 lines 1-4 “In some embodiments, the broadcast information may be received, and the vocabulary updated accordingly, based on a determined location being in proximity to a predefined location,”)
access first audio data associated with a radio station, (Ingebretsen ¶ 0247 “In these cases, various stations may be available over the internet or cellular connection (e.g., via one or more streaming technologies based on various audio format). For example, a respective station that is available over traditional FM band may also be available over an internet connection, such as 104.1 FM, “WABC FM.” In response to a user request such as “play WABC FM” or “play 104.1 WABC FM,” the device may make a determination whether to play the station over traditional FM band or over cellular or internet connection. In particular, electronic device 802 may receive the media information […]and may then transfer the media to the computing device for playback through speakers within the vehicle. In cases where the radio transceiver is within range of the radio transmitter for the radio station, the system may verify a user preference as to whether to play stations over traditional AM/FM band or internet or cellular connection when all options are available,” teaching accessing audio media information related to a radio station)
wherein the geo-location indicator represents a location of the ego-machine in the environment; and […] (Ingebretsen ¶ 0059 lines 15-19 “Device 200 optionally includes […] a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information concerning the location […] of device 200.”)
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the television-based information used for conversation generation of Yamazaki with the simple addition of Ingebretsen’s receiving of location-based radio station audio information. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests, to improve safety, and to more effectively leverage radio broadcast matter in speech interaction (Ingebretsen ¶ 0004)
The combination of Yamazaki and Ingebretsen does not teach:
[…] second […] second […]
[…] of a summary […]
Within the same field of endeavor as Yamazaki and Ingebretsen, Webster teaches:
[…] a summary […] (Webster ¶ 0046 lines 5-14 “the output content 208 represents the output of the voice assistant device 108 […] In one or more implementations, the output content 208 is configured as data for audible output by the voice assistant device 108, e.g., audible speech. Additionally or alternately, the output content 208 may be configured as data for display by the voice assistant device 108 or another device. […] output content 208 may be a […] summary,” teaching a summarization of data to be presented)
Yamazaki, Ingebretsen, and Webster are all considered analogous because they all relate to assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the media-based information used for conversation generation of Yamazaki and Ingebretsen’s receiving of location-based radio station audio information with the simple addition of Webster’s summarization of data to be presented. This modification would be made with a reasonable expectation of success as motivated by improving conciseness and usefulness of data presented to the operator.
The combination of Yamazaki, Ingebretsen, and Webster does not teach:
[…] second […] second […]
Within the same field of endeavor as Yamazaki, Ingebretsen, and Webster, Shinkai teaches:
[…] provide one or more second […] prompts […] as second input into the one or more language models […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki, Ingebretsen, Webster, and Shinkai are all considered analogous because they all relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Regarding Claim 14, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to: […] provide one or more […]text prompts comprising a representation of the audio data as […] input into the one or more language models; (Yamazaki Pg 3 ¶ 7 lines 4-8 “by using the classifier on the cloud, the usage history of the information processing device used by the user […] for example, a DVD playback title, a TV program watched). It is possible to make the classifier learn the history of contents, […] as preference information,” and Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, […] using preference information.”) teaching the use of media content, and specifically music (audio data) as preference information input into the machine learning model, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
and cause presentation, using the display device or sound device associated with the operator or occupant of the ego-machine, of […] the representation of the audio data. (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, […] using preference information.”)
Yamazaki does not teach:
[…] based at least on a geo-location indicator, access first audio data
associated with a radio station,
wherein the geo-location indicator represents a location of the ego-machine;
and […]
[…] second […] second […]
[…] a summary of […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…] based at least on a geo-location indicator, (Ingebretsen ¶ 0244 lines 1-4 “In some embodiments, the broadcast information may be received, and the vocabulary updated accordingly, based on a determined location being in proximity to a predefined location,”)
access first audio data associated with a radio station, (Ingebretsen ¶ 0247 “In these cases, various stations may be available over the internet or cellular connection (e.g., via one or more streaming technologies based on various audio format). For example, a respective station that is available over traditional FM band may also be available over an internet connection, such as 104.1 FM, “WABC FM.” In response to a user request such as “play WABC FM” or “play 104.1 WABC FM,” the device may make a determination whether to play the station over traditional FM band or over cellular or internet connection. In particular, electronic device 802 may receive the media information […]and may then transfer the media to the computing device for playback through speakers within the vehicle. In cases where the radio transceiver is within range of the radio transmitter for the radio station, the system may verify a user preference as to whether to play stations over traditional AM/FM band or internet or cellular connection when all options are available,” teaching accessing audio media information related to a radio station)
wherein the geo-location indicator represents a location of the ego-machine; and […] (Ingebretsen ¶ 0059 lines 15-19 “Device 200 optionally includes […] a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information concerning the location […] of device 200.”)
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the television-based information used for conversation generation of Yamazaki with the simple addition of Ingebretsen’s receiving of location-based radio station audio information. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests, to improve safety, and to more effectively leverage radio broadcast matter in speech interaction (Ingebretsen ¶ 0004)
The combination of Yamazaki and Ingebretsen does not teach:
[…] second […] second […]
[…] a summary of […]
Within the same field of endeavor as Yamazaki and Ingebretsen, Webster teaches:
[…] a summary of […] (Webster ¶ 0046 lines 5-14 “the output content 208 represents the output of the voice assistant device 108 […] In one or more implementations, the output content 208 is configured as data for audible output by the voice assistant device 108, e.g., audible speech. Additionally or alternately, the output content 208 may be configured as data for display by the voice assistant device 108 or another device. […] output content 208 may be a […] summary,” teaching a summarization of data to be presented)
Yamazaki, Ingebretsen, and Webster are all considered analogous because they all relate to assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the media-based information used for conversation generation of Yamazaki and Ingebretsen’s receiving of location-based radio station audio information with the simple addition of Webster’s summarization of data to be presented. This modification would be made with a reasonable expectation of success as motivated by improving conciseness and usefulness of data presented to the operator.
The combination of Yamazaki, Ingebretsen, and Webster does not teach:
[…] second […] second […]
Within the same field of endeavor as Yamazaki, Ingebretsen, and Webster, Shinkai teaches:
[…] and provide one or more second […] prompts […] as second input into the one or more language models; […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki, Ingebretsen, Webster, and Shinkai are all considered analogous because they all relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view Ingebretsen and further in view of Dasher et al (US 20250085131, hereinafter “Dasher”) and Shinkai.
Regarding Claim 7, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki does not teach:
wherein the one or more processing units further to: access, from one or more data sources, destination or travel route information
associated with a destination or travel route of the ego-machine;
and provide one or more second text prompts comprising a representation of the destination or travel route information as second input into the one or more language models
to generate a summarized representation of the destination or travel route information.
Within the same field of endeavor as Yamazaki, Dasher teaches:
wherein the one or more processing units further to: access, from one or more data sources, destination or travel route information associated with a destination or travel route of the ego-machine;
and provide one or more […] text prompts comprising a representation of the destination or travel route information as […] input into the one or more language models (Dasher ¶ 0055 lines 1-6 “LLM agents are prompted with contextual data in some embodiments. The contextual data is provided by a mapping application and by searching for related information within a context of a location on a map. The contextual data is determined through user selection, user location search or a zoom to location performed by a user,” and ¶ 0056 lines 1-8 “In some embodiments, prompts are dynamically generated. The prompts are personalized and context-aware. The prompts result in output (e.g., audio output, different pictures, and the like) that is dynamic and does not persist. More specifically, dynamic prompts are generated based on location of a moving object (e.g., car), a projected path of the moving object (e.g., based on start and/or destination addresses),” emphasis added)
to generate a summarized representation of the destination or travel route information. (Dasher ¶ 0057 lines 5-13 “The context data is derived from mapping applications considering information, such as at least one of map location data, embedded map POI data, external data such as weather or traffic data, data learned from map POI data (for example, by reading a URL embedded into POI data, by following links, and by summarizing the information), combinations of the same, or the like. For embodiments including a generative video system, the system is configured to generate a sequence of images, e.g., navigation images and direction overlays,” emphasis added)
Yamazaki and Dasher are considered analogous because they both relate to artificial intelligence content generation systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the learning models used for conversation generation of Yamazaki with the simple addition of Dasher’s LLM agent prompted with contextual data of projected path and destination address. This modification would be made with a reasonable expectation of success as motivated by achieving a more immersive and interactive user experience with rich and contextually relevant presentation (Dasher ¶ 0055 lines 15-18.)
The combination of Yamazaki and Dasher does not teach:
[…] second […] second […]
Within the same field of endeavor as Yamazaki and Dasher, Shinkai teaches:
[…] provide one or more second […] prompts […] as second input into the one or more language models […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Yamazaki, Dasher, and Shinkai are all considered analogous because they all relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Shinkai teaches a system implemented using a robot (Shinkai Col 5 lines 40-45 “Referring to FIG. 2, the technical concept according to the present embodiment will be further described. FIG. 2 is a diagram illustrating an aspect in which interactive home-appliance system 20 is used. In an aspect, home-appliance system 20 is implemented as cleaning robots 211, 221, which are an embodiment of the home appliance.”)
Dasher teaches real-time streaming (Dasher ¶ 0174 “In some embodiments, the system 100 is configured to include ultra-low latency transport for encoded video for a sequence of AI-generated images. The system 100 is configured to implement such functionality in a manner similar to low latency streaming of video game cloud rendering”)
Dasher also teaches a system for generating synthetic data, and a system for generating synthetic data using AI (Dasher ¶ 0066 “generative AI herein refers to a type of artificial intelligence that is configured to generate unique artifacts by analyzing data. The artifacts are used for a variety of purposes. Generative AI creates at least one of […] synthetic data.”)
Homma (US 20220196423) teaches a safety drive assist apparatus which performs a 5-level evaluation of fatigue level in drivers, describing multiple fatigue levels
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZACHARY E GLADE whose telephone number is (703)756-1502. The examiner can normally be reached 4-5-9 7:30-16:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at (571) 270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZACHARY E. F. GLADE/Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664
