DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the application filed on 11/01/2023 and the amendments and response filed 8/28/2025.
Claims 1-20 have been amended.
No claims have been added.
No claims have been cancelled.
Claims 1-20 are currently pending and have been examined.
Information Disclosure Statement
The information disclosure statement(s) (IDS(s)) submitted on 2/15/2024 has been received and considered.
Response to Amendment
Applicant’s amendments to the Abstract, Specification, Drawings, and Claims have overcome each and every objection and 112 Markush Grouping rejection previously set forth in the Non-Final Office Action mailed 6/17/2025.
Response to Arguments
Applicant’s arguments, see pages 15-18, filed 8/28/2025, with respect to the rejection(s) of claim(s) 1-20 under 35 USC 102 and 103 have been fully considered and are persuasive regarding Yamazaki not teaching the newly amended elements of “one or more text prompts,” “one or more subsequent text prompts,” and “one or more language models hosted on or accessible to the ego-machine.” Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made as necessitated by amendment in view of Yamazaki (WO 2021148903) and Ingebretsen (US 20230344537).
The argument that Yamazaki does not teach “mapping computed alertness level to a natural language description of the computed alertness level,” is not persuasive. Yamazaki Pg 3 ¶ 7 line 1 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker,” demonstrates the generation of a natural language description of the alertness (“sleepy”) generated from a result of the biosensor detection of the level of alertness including “drowsiness” in Pg 5 ¶ 8 lines 1-6, demonstrating by example a mapping of the sensor result to a natural language descriptor as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”
Claim Objections
Claim 12 is objected to because of the following informalities: line 3 of the amended claim reads “[…] text prompts provided provided as […]” where “provided” is redundant. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 3, 8, 9, 11, 15-16, 18, and 20 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki et al (WO 2021148903, hereinafter “Yamazaki,” all excerpts and citations taken from the attached machine translation) in view of Ingebretsen et al (US 20230344537, hereinafter “Ingebretsen”).
Regarding Claim 1, Yamazaki describes:
One or more processors comprising: (Yamazaki Pg 10 ¶ 4 line 1 “The control unit 21 includes a processor 60,”)
one or more processing units to: receive image data (Yamazaki Pg 4 ¶ 2 line 1 “The calculation unit can perform image analysis,”)
representing one or more portions of an operator of an ego-machine, (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,”)
the image data generated using one or more sensors (Yamazaki Pg 4 ¶ 1 line 1 “The image pickup device has a first image pickup device and a second image pickup device.,”)
of an ego-machine traversing an environment; (Yamazaki Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
based at least on the image data, generate a representation of a computed alertness level of the operator; […] (Yamazaki Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination of a user’s level of alertness that includes “drowsiness”)
[…] comprising the representation of the computed alertness level as an input (Yamazaki Pg 5 ¶ 6 lines 1-4 “In addition, the biosensor can detect the biometric information of the driver. The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93,”)
into one or more […] models (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching the input of the biosensor data into one of several machine learning models accessible to the device)
to generate one or more natural language characters (Yamazaki Pg 4 ¶ 5 line 1 “The conversation information generation unit can generate the first conversation information […]” the conversational information being analogous to natural language characters, as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”)
based at least on the computed alertness level of the operator; (Yamazaki Pg 4 ¶ 5 lines 1-2 “[…] based on the biological information […]”)
and cause presentation, using a display device or sound device associated with the operator of the ego-machine, of a representation of the one or more natural language characters. (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
Yamazaki does not teach:
[…] provide one or more text prompts […]
[…] language […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…] provide one or more text prompts […] into one or more language models […]
(Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 3, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further describes:
wherein the one or more processing units are further to generate the representation of the computed alertness level of the operator based at least on mapping the computed alertness level to a natural language description of the computed alertness level, and including the natural language description of the computed alertness level of the operator in the one or more text prompts. (Yamazaki Pg 3 ¶ 7 line 1 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker,” demonstrating the generation of a natural language description of the alertness (“sleepy”) generated from a result of the biosensor detection of the level of alertness including “drowsiness” as previously cited in Pg 5 ¶ 8 lines 1-6, demonstrating by example a mapping of the sensor result to a natural language descriptor as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Regarding Claim 8, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further describes:
wherein the one or more processors is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; (Yamazaki Pg 4 ¶ 7 “Therefore, one aspect of the present invention can be referred to as an information processing system or an automatic driving support system using the above-mentioned information processing device.”)
a perception system for an autonomous or semi-autonomous machine; (Yamazaki Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
or a system implemented using a robot.
Regarding Claim 9, Yamazaki describes:
A system comprising one or more processing units (Yamazaki Pg 10 ¶ 4 line 1 “The control unit 21 includes a processor 60,”)
to cause presentation, using a device associated with an operator of an ego-machine, (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
of a representation of one or more natural language characters (Yamazaki Pg 4 ¶ 5 line 1 “The conversation information generation unit can generate the first conversation information […]” the conversational information being analogous to natural language characters, as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”)
generated using one or more […] models hosted on or accessible to the ego-machine to process […] (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching conversation generation by several machine learning models accessible to the device based on the biosensor data)
comprising a representation of a computed alertness level of an operator (Yamazaki Pg 4 ¶ 5 lines 1-2 “[…] based on the biological information […]” and Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination that the user’s alertness level includes “drowsiness”)
of an ego-machine. (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,” and Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,” emphasis added)
Yamazaki does not teach:
[…] language […]
[…] a text prompt […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…]using one or more language models […] to process a text prompt […] (Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 11, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further describes:
wherein the one or more processing units are further to generate the representation of the computed alertness level of the operator based at least on mapping the computed alertness level to a natural language description of the computed alertness level, and include the natural language description of the computed alertness level of the operator in the text prompt. (Yamazaki Pg 3 ¶ 7 line 1 “In FIG. 1D, as a result of the biosensor detecting the driver's drowsiness, the driver is asked "Are you sleepy?" By generating conversation information 93 and asking the driver from the speaker,” demonstrating the generation of a natural language description of the alertness (“sleepy”) generated from a result of the biosensor detection of the level of alertness including “drowsiness” as previously cited in Pg 5 ¶ 8 lines 1-6, demonstrating by example a mapping of the sensor result to a natural language descriptor as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
Regarding Claim 15, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further describes:
wherein the one or more processors is comprised in at least one of:
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; (Yamazaki Pg 7 ¶ 3 “Step S103 is a step of displaying the object 92 on the object 91. Augmented reality or mixed reality can be realized by superimposing the object 92 on the image visually recognized by the eyeglass function of the wearable device 10. Therefore, the object 92 as shown in FIG. 1D can be displayed via the wearable device 10.”)
a system for generating synthetic data; or
a system for generating synthetic data using AI.
Regarding Claim 16, Yamazaki describes:
A method comprising: receiving a representation of a computed alertness level (Yamazaki Pg 5 ¶ 8 lines 1-6 “As an example, an infrared sensor can be used as the biosensor. […] For the detection of drowsiness, the number of times the eyelids are opened and closed can be added to the judgment condition. Therefore, the second imaging device can be included in one of the biosensors because it can detect the movement of the user's eyes, the degree of opening of the eyelids, and the like,” teaching a determination of a user’s level of alertness that includes “drowsiness”)
of an operator of an ego-machine; (Yamazaki Pg 4 ¶ 1 lines 2-4 “The second image pickup device captures a second image for detecting the movement of the user's eyes, the degree of eyelid opening, the number of blinks of the eyes, and the like,” and Pg 4 ¶ 5 lines 6-9 “Further, the conversation information generation unit can generate conversation information by combining preference information and [...] driver information, information captured by an in-vehicle imaging device,”)
receiving a representation of one or more interests of the operator; (Yamazaki Pg 3 ¶ 7 line 1 “The conversation information generation unit has a classifier that learns the user's preference information,” Pg 3 ¶ 7 lines 4-8 “by using the classifier on the cloud, the usage history of the information processing device used by the user (when the information processing device is incorporated in a home appliance, for example, a DVD playback title, a TV program watched). It is possible to make the classifier learn the history of contents, the stored contents of the refrigerator, the operation history of the dishwasher, etc. as preference information,” the user history preference information stored on the cloud being analogous to a interests of the user)
[…] comprising the representation of the computed alertness level and the representation of the one or more interests of the operator as at least a portion of an input (Yamazaki Pg 5 ¶ 6 lines 1-4 “In addition, the biosensor can detect the biometric information of the driver. The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93,”)
into one or more […] models (Yamazaki Pg 5 ¶ 4 lines 1-3 “The information processing device shown in FIG. 1C is an interactive voice response device 80 provided with a biosensor. The voice automatic response device 80 may be paraphrased as an AI speaker. The interactive voice response device 80 includes a speaker 81, a microphone 82, and a biosensor 83,” and Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. Neuro-Linguistic Programming (NLP), Deep learning using a neural network, and the like can be used for conversation generation,” teaching the input of the data into one of several machine learning models)
to generate one or more natural language characters (Yamazaki Pg 4 ¶ 5 line 1 “The conversation information generation unit can generate the first conversation information […]” the conversational information being analogous to natural language characters, as interpreted according to the specification ¶ 0027 “natural language characters (e.g., an English phrase)”)
based at least on the computed alertness level of the operator and the one or more interests of the operator; (Yamazaki Pg 4 ¶ 5 lines 1-2 “The conversation information generation unit can generate the first conversation information based on the biological information and the preference information,” and Pg 8 ¶ 2 lines 1-6 “Step S112 is a step in which the conversation information generation unit generates conversation information 93a using topic information, biological information, and preference information possessed by the classifier. The classifier preferably extracts information from the topical information that is likely to activate the driver's brain. It is preferable that the conversation information 93a corresponding to the alert or warning is generated from the biometric information. As an example, the conversation information 93a is generated using information that is likely to activate the driver's brain using topical information,”)
and causing presentation, using a device associated with the operator of the ego-machine, of a representation of the one or more natural language characters. (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
Yamazaki does not teach:
[…] providing one or more text prompts […]
[…] language […]
Within the same field of endeavor as Yamazaki, Ingebretsen teaches:
[…] provide one or more text prompts […] into one or more language models […]
(Ingebretsen ¶ 0220 lines 1-3 “Natural language processing module 732 receives the candidate text representations (e.g., text string(s) or token sequence(s)),” teaching the use of text-based inputs into a natural language processing module (language model))
Yamazaki and Ingebretsen are considered analogous because they both relate to driving assistance artificial intelligence systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversation information and biometric information input into the conversation information generation unit of Yamazaki with the simple substitution of Ingebretsen’s text representation input for the unspecified format of Yamazaki’s input and Ingebretsen’s natural language processing module for the conversation information generation unit of Yamazaki. This modification would be made with a reasonable expectation of success as motivated by better interpretation of user requests and improved safety (Ingebretsen ¶ 0004).
Regarding Claim 18, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further describes:
wherein the presentation, at the device associated with the operator of the ego-machine, of the representation of the one or more natural language characters generated by the one or more language models comprises presentation, using a sound device, (Yamazaki Pg 4 ¶ 5 lines 2-3 “[…] The speaker can output the first conversation information”)
of a first phrase utterance that initiates a conversation with the operator (Yamazaki Pg 8 ¶ 2 lines 8-10 “The conversation information generation unit preferably generates question-type conversation information 93a for which the driver needs a reply,”)
based at least on the one or more interests. (Yamazaki Pg 8 ¶ 2 lines 1-6 “Step S112 is a step in which the conversation information generation unit generates conversation information 93a using topic information, biological information, and preference information possessed by the classifier. The classifier preferably extracts information from the topical information that is likely to activate the driver's brain. It is preferable that the conversation information 93a corresponding to the alert or warning is generated from the biometric information. As an example, the conversation information 93a is generated using information that is likely to activate the driver's brain using topical information,”)
Regarding Claim 20, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further describes:
wherein the method is performed by at least one of: a system for performing deep learning operations; (Yamazaki Pg 10 ¶ 1 lines 1-3 “Further, the conversation information generation unit 16 can generate conversations based on the classification data of the classifier. […] Deep learning using a neural network, and the like can be used for conversation generation,”)
a system for performing real-time streaming;
a system implemented using an edge device;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources. (Yamazaki Pg 3 ¶ 7 line 1 “The conversation information generation unit has a classifier that learns the user's preference information,” Pg 3 ¶ 7 line 4 “by using the classifier on the cloud,)
Claim(s) 2, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view of Ingebretsen and further in view of Shinkai et al (US 10224060, hereinafter “Shinkai”) and Sawai (US 20230186653, hereinafter “Sawai”)
Regarding Claim 2, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to provide one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the one or more natural language characters […] as a subsequent input into the one or more language models […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information”)
Yamazaki does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
[…] to generate one or more second natural language characters representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Regarding Claim 10, the combination of Yamazaki and Ingebretsen teaches the elements of claim 9 as described above. Yamazaki further teaches:
wherein the one or more processing units are further to provide one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the one or more natural language characters […] as at least a portion of an input into the one or more language models […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information”)
Yamazaki does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
[…] to generate one or more second natural language characters representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Regarding Claim 17, the combination of Yamazaki and Ingebretsen teaches the elements of claim 16 as described above. Yamazaki further teaches:
further comprising providing one or more subsequent text prompts comprising a representation of a phrase utterance that represents an operator response by the operator to the presented representation of the one or more natural language characters […] as a subsequent input into the one or more language models […] (Yamazaki Pg 6 ¶ 1 line 11-13 “When the microphone […] detects the driver's voice (conversation information 94), the conversation information 94 is converted into linguistic data by the conversation information generation unit, and the linguistic data can update the preference information”)
Yamazaki does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
[…] to generate one or more second natural language characters representing a generated response to the operator response.
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] to generate one or more second natural language characters representing a generated response to the operator response. (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on process of determining the user’s response and outputting a second utterance. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the conversion of the user’s conversation information into linguistic data to update the preference information of Yamazaki with the simple addition of Shinkai’s second utterance in response to the user’s reply. This modification would be made with a reasonable expectation of success as motivated by easing the user’s psychological state with the content of the reply (Shinkai Col 4 lines 65-67.)
The combination of Yamazaki and Shinkai does not teach:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […]
Within the same field of endeavor as Yamazaki and Shinkai, Sawai teaches:
[…] and a representation of a subsequent computed alertness level computed subsequent to the operator response […] (Sawai ¶ 0109 and ¶ 0110 lines 1-8 “[0109] First, the driver monitoring unit 233 determines whether or not the alertness level of the driver has fallen, based on the level of contribution of the driver 40 to driving (step S301). When a low level of contribution of the driver 40 to driving has continued for a predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has fallen (step S301—Yes). When a low level of contribution of the driver 40 to driving has not continued for the predetermined monitoring reference time, the driver monitoring unit 233 determines that the alertness level of the driver 40 has not fallen (step S301—No).
[0110] When the alertness level of the driver has fallen, the control unit 235 acts on the driver 40 to increase the level of contribution to driving via the UI 4 (step S303), and the series of processing steps is complete. The action may be notifying the driver 40 of the action information via the UI 4. The action information may be, for example, display information displayed on the UI 4, or audible information output from the UI 4,” teaching generation of audio data, similar to Yamazaki, after a continued period of inattentiveness, analogous here to a subsequent measurement of fatigue to Yamazaki’s determination.”)
Yamazaki and Sawai are both considered analogous because they both relate to alertness warning systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker including a determination of fatigue. Sawaii presents an alertness monitoring system which operates over a continued period of time from a first measurement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fatigue determination prompting a conversation of Yamazaki with the addition of Sawai’s continued alertness monitoring resulting in audible information output by the UI. This modification would be made with a reasonable expectation of success as motivated by improving the effectiveness of the fatigue alleviation by continued monitoring, as would be obvious to one of ordinary skill in the art.
Claim(s) 4, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yamazaki in view Ingebretsen and further in view of Shinkai.
Regarding Claim 4, the combination of Yamazaki and Ingebretsen teaches the elements of claim 1 as described above. Yamazaki further teaches:
wherein one or more […] text prompts provided as […] input to the one or more language models further include at least one of: (Yamazaki Pg 5 ¶ 6 line 1-6 “The conversation information generation unit can select the detected biometric information and the preference information from the classifiers of the conversation information generation unit, and combine the biometric information and the preference information to generate the conversation information 93. The preference information may be selected from a classification having a large number of registered items, or may be selected from a classification having a small number of registered items,” describing preference information being used as a prompt, as applies to the text representation input and natural language processing module of Ingebretsen ¶ 0220 lines 1-3 previously cited)
personalized information of the operator, (Yamazaki Pg 6 ¶ 5 line 8-10 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information.”)
a natural language instruction to start a conversation with the operator that aligns with the operator's interests, (Yamazaki Pg 6 ¶ 5 line 8-12 “conversation information 93 can be generated using music, TV programs, food, recently taken pictures, usage history of home appliances such as the contents of the refrigerator, etc. using preference information. The conversation information generation unit preferably generates question-type conversation information 93 for which the driver needs a reply.”)
a zero-shot, one-shot or few shot examples of representative inputs or outputs, (Yamazaki Pg 7 ¶ 8 “In step S011, Internet news and the like acquired by the vehicle control unit via satellite or wireless communication can be collected as topic information. In addition, the in-vehicle imaging device included in the vehicle monitoring unit can collect images captured from the driving vehicle. For example, it is possible to collect topical information such as the vehicle type and speed of vehicles passing each other, clothes of pedestrians, and images of vehicles driving abnormally.”)
or an instruction to send a control signal to cause a particular action of the ego-machine. (Yamazaki Pg 6 ¶ 4 line 6-9 “Therefore, by giving the driver information detected by the wearable device 10 to the vehicle, it is possible to suppress the occurrence of accidents due to looking away driving, dozing driving, and the like. Semi-automatic driving or automatic driving can be canceled based on the driver information. The driver information is also given to the conversation information generation unit.”)
Yamazaki does not teach:
[…] subsequent […] subsequent […]
Within the same field of endeavor as Yamazaki, Shinkai teaches:
[…] wherein one or more subsequent text prompts provided as subsequent input […] (Shinkai Col 4 lines 44-67 “In an aspect, for example, home-appliance system 20 outputs voice through utterance unit 21 (first utterance). User 11 replies in response to the voice. Voice input unit 22 accepts input of the reply by user 11. Voice input unit 22 converts the reply into a voice signal. […] In control unit 27, determination unit 23 determines the psychological state of user 11 based on the signal and data stored in storage unit 24. This determination is made based on, for example, data stored in storage unit 24 in advance. […] Determination unit 23 transmits a signal based on the result of the determination to utterance unit 21. Utterance unit 21 outputs voice based on the signal (second utterance). Listening to the voice (second utterance) output by home-appliance system 20, user 11 may ease the psychological state with its content,” teaching the generation of a second natural language character, as a natural language response interpreted based on the specification ¶ 0027 as “a natural language response (e.g., one or more characters, words, etc.)”)
Yamazaki and Shinkai are considered analogous because they both relate to conversational artificial intelligence systems. Yamazaki presents a “conversation system” prompting the user to reply with “conversation information,” implying but not explicitly specifying a second and/or ongoing generation of conversation information to be output by the speaker. Shinkai teaches this follow-on proc