DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In response to the office action from 5/8/2025, the applicant has submitted an amendment, filed 8/7/2025, amending claims 1, 3-14, cancelling claim 2, while arguing to traverse the prior art and 101 rejections. Applicant’s arguments have been fully considered but are moot with respect to new grounds of rejections further in view of XIAO GONGTING (CN 109795319) mandated by the latest amendments and for the reasons explained in the response to arguments.
Response to Arguments
Following a broad overview of the latest amendments on page 9 the first ¶, from the second ¶ on page 9 to the end of the first ¶ on page 18 the previous 35 U.S.C. 101 is discussed.
Before addressing the applicant’s comments and arguments, it should be noted that the dependent claims 9 and 10 were not subject to Alice 101 rejection. This does provide a recipe to overcome that rejection.
Following a broad overview of the latest amendments on page 9 paragraph 1, from the second paragraph of that page to the end of the first paragraph on page 18, the previous 101 rejection is discussed.
Following quotation of some unrelated cases on page 11 paragraph 1, a copy of the claim on the remainder of the page, a copy of some sections of applicant’s disclosure on pages 12-13, it is concluded that “human mind is not equipped to “obtain using a natural language-based communication interface, an indication from the operator of an operator’s current fatigue level… wherein the query includes a structured prompt …” which corresponds to the latest amended limitation of claim 1. Then on page 15 the third paragraph lines 2-5 it is concluded that: “Applicant respectfully submits that the amended claims as a whole integrate the recited abstract ideas into a practical application and are therefore eligible for patentability under Mayo Step 2A, Prong II”. And on page 16 the first sentence it is further concluded: “Applicant respectfully submits that the present invention is directed to improvements for gathering more (and more meaningful) information regarding the operator’s fatigue level” …”.
This scenario can be achieved by a human who can easily determine how tired another person is by simply asking them, and given their tone, he can probe their fatigue level, and further if any remedy is suggested to them to determine how that remedy might have worked by asking them a second time the same question.
As regards to “practical application” and resulting “improvements” resulting from the claims, unfortunately these limitations as drafted do not adhere to the USPTO standards: i.e., please see: “To make the determination of whether these claims are directed to an improvement in existing computer technology, the court looked to the teachings of the specification. Specifically, the court identified the specification’s teachings that the claimed invention achieves other benefits over conventional databases, such as increased flexibility, faster search times, and smaller memory requirements” (Enfish memorandum of 5/19/2016); and/or “An “improvement in computer-related technology” is not limited to improvements in the operation of a computer or a computer network per se, but may also be claimed as a set of “rules” (basically mathematical relationships) that improve computer-related technology by allowing computer performance of a function not previously performable by a computer” (MCRO memorandum of 11/2/2016).
Page 17 the first paragraph lists a few court cases without drawing any specific one on one mapping between those and the instant application. On page 17 the last paragraph it is however recited that the “current ambient conditions” “may be obtained using the one or more environmental sensors”.
If by this recitation, it is attempted to suggest a device (i.e., the “sensor”) is required for the claim limitations, it should be mentioned that unless the said “sensors” were invented by the instant application, the only way these limitations involving the “sensors” would help the claims to recite significantly more would have been for the said limitations to improve the functionality of the said “sensors” (please see the quoted memorandums of Enfish and McRO above).
From page 18 the 2nd paragraph to the end of page 33 arguments directed at why the primary reference Yan Bin et al. fails to teach the latest amendments.
Since Yang Bin et al. is not used for the latest amendments, please visit the new office action further in view of XIAO GONGTING to see how the amendments are addressed.
On page 34 the first paragraph it is argued that the dependent claims “are allowable based on their dependence upon an allowable base claim”.
Since applicants have not argued the merits of these dependent claims, but assert patentability solely through their dependence on the allegedly patentable parent claims, they stand or fall with said parent claims and hence no further response to applicant’s arguments is necessary.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, 11-14 stand rejected:
The independent claims 1, 11 and 12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite about “monitoring a fatigue level” “of an operator” (e.g., a driver of an “intelligent vehicle system”), based on a verbal communication with its “operator” (sp. Par. 0024: “periodically ask the operator for an indication of their current fatigue level”), via a “communication interface” (sp. Par. 0079: “interface” “to provide audio output to the operator and to receive verbal responses from the operator”). The response of the “operator” is termed as an “indication” “or their current fatigue level” “provided using natural language” which is “processed” using an “NLP” “unit” “to determine” “one or more data points indicative” or the “fatigue level” (Sp. Par. 0030: “For instance” using “a machine learning” “to process” the “NLP” “indications from an operator in order to determine data points”” by “extract[ing] a (numerical) value”). The method further uses “one or more environmental sensors” to probe “current ambient conditions” (Sp. Par. 0045: “such as temperature, lightning conditions, humidity, air quality, and the like”), and use these “to reduce operator fatigue” (Sp. Par. 0038: “For example” “when” “operator is feeling too cold the processing may determine” “turning on a heater of the system”). Finally a “set of rule” are “generat[ed]” so that “when similar ambient conditions” “are” “encountered” “the respective rule” associated with the “ambient conditions” and their “corresponding” “actions” are “performed”.
These limitations in general can be almost all carried out by a human especially since even the “fatigue level” is basically determined based on interacting with the “operator”. Here a human can easily determine how tired another person is by simply asking querying them, and given their tone, he can probe their fatigue level; e.g., even assign a numerical value on a scale of ten.
The usage of “sensors” to determine the “ambient conditions” which are potentially responsible for the operator fatigue are recited at a very high level of generality without any meaningful limitations on their functions; i.e., the corresponding steps as formulated via claim 1 limitations 4+ are recited at a very high level of generality without really specifying any specific technique, which makes the “sensors” extra solution activity; that being so since one can sense fatigue to be attributed to heat and humidity without knowing the exact temperature and/or humidity level and even take actions to help alleviate that, e.g. by recommending an operator to turn on an AC.
Finally the last step which is to basically store any actions taken for any specific ambient condition personalized to an operaor, and using them for future potential similar ambient conditions and their corresponding operator fatigue reduction. Here again a human can learn intuitively and by experience certain actions pertaining to certain specific fatigue believed to have arisen from e.g. heat and humidity for a certain operator and should similar conditions happen to alleviate the operator.
These judicial exceptions as indicated are not integrated into a practical application. In particular, the claims only recite “sensors” “control circuit” and the “NLP” “unit” (claim 1), “computer readable storage medium” “software” “data processor” (claim 11), “apparatus” (claim 12) as additional elements to perform the listed limitations above. That is other than the recitation of the said elements, nothing in the said claims preclude the limitations from being performed in the mind as shown above. Therefore the claims limitations, as drafted, under their broadest reasonable interpretations, cover performance of the limitations in the mind.
The judicial exceptions are not integrated into a practical application. The additional elements (e.g. the “sensors”, “processor” “medium”) are recited at a high-level of generality (e.g., no specific sensor and/or processor which amount to generic sensors and processors and media), and thus they do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims are thus directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The functions of e.g., the “sensors” “control circuit” “processors” to do generic functions (e.g., data gathering pertaining to ambient conditions) and correlating them with statements of the user to perform some remedial actions reduce their functions to generic routine functions. The claims are thus not patent eligible.
Regarding claims 2-4, 6, 13-14 the action of prompting an operator to determine his condition and/or further prompting to determine if a recommendation to him has worked and potentially validating them do not require any particular device, machine and/or method and can be carried out by a human.
Claims 7-8 correspond to the last 2 limitations of claim 1 and similar scenario described above for those limitations are applicable here.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 6-7, 9-12, 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over YANG BIN (CN 113223270(A)), and further in view of XIAO GONGTING (CN 109795319 (A)).
Regarding claim 1, YANG BIN et al. do teach a method of monitoring a fatigue level of an operator (Title, Abstract sentence 1: “The invention discloses a fatigue driving monitoring”),
the method comprising:
obtaining, using a natural language-based communication interface, an indication from the operator of an operator’s current fatigue level, wherein the indication is provided using natural language (¶ n0114 lines 1+: “the monitoring system detects that the driver’s fatigue value is 3” (obtaining an indication of a “driver’s” (operator’s) current fatigue level) “the intelligent voice module responds” (using a natural language-based communication interface) “broadcasts the driver’s real-time fatigue value” (where the indication is obtained in natural language and indicates a current “fatigue value” (fatigue level) of the operator);
providing the indication to a natural language processing (NLP) unit, the NLP unit performing natural language-based processing on the indication to determine a set of one or more data points indicative of the operator’s current fatigue level (¶ n0114 lines 1+: “the monitoring system” (a natural language processing unit) “detects” (determines) “that the driver’s fatigue value is 3” (from the indication a data point “3” describing the fatigue level) “the intelligent voice module responds” “broadcasts the driver’s real-time fatigue value” “You are now severely fatigued”);
obtaining, from one or more environmental sensors, data indicative of one or more current ambient conditions within a system, wherein the system is a collaborative system that is operated through a combination of one or more actions of the operator and one or more additional actions performed by a control circuit of the system, the one or more ambient conditions thereby being associated with the operator’s current fatigue level (¶ n0119 lines 3+: “A high concentration of CO2” (an ambient environmental condition) “increase their sleepiness” (associated with the operator’s fatigue level) “The CO2” “concentration monitoring device” (obtaining from one or more environmental sensors) “the CO2” “concentration in the cab” (data indicative of the current ambient conditions) “when the monitored CO2 concentration is higher than 1000 ppm, the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” “please open the window for ventilation” (i.e., a collaborative system which depends on actions from the operator and those mandated by a control circuit); ¶ n0120 lines 2+: “temperature” (another ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue) “The air temperature and humidity monitoring device” (a sensor) “monitors the temperature and humidity in the air in the cab” “When the monitored air temperature is higher than 35 [degrees] C” (obtains an environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (acts to adjust the temperature));
determining, using the determined set of one or more data points indicative of the operator’s current fatigue level and the associated one or more current ambient conditions, the one or more additional actions to be performed by the control circuit of the system to reduce operator fatigue (¶ n0119 lines 3+: “A high concentration of CO2” “increase their sleepiness” (fatigue measured by e.g., “fatigue value” (the one or more data points)) “The CO2” “concentration monitoring device” “monitors” “the CO2” “concentration in the cab” (and data indicative of the current ambient conditions) “When the monitored CO2 concentration is higher than 1000 ppm” “the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” (takes one or more actions) “please open the window for ventilation” (to help reduce operator fatigue); ¶ n0120 lines 2+: “temperature” (an ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue measured by “fatigue value” (the one or more data points)) “When the monitored air temperature is higher than 35 [degrees] C” (an obtained environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (determines and performs one or more actions to reduce operator’s fatigue level attributed to higher than 35 [degrees] C)); and
generating a respective rule personalized to the operator that can be automatically applied by the control circuit when similar ambient conditions are subsequently encountered within the system, the respective rule comprising an input set of ambient conditions and a corresponding one or more additional actions to be performed when the input set of ambient conditions is encountered (¶ n0006 last 3 lines: “the fatigue value real-time monitoring system is used to identify the fatigue characteristic posture of the driver” (an operator’s personalized rule generated) “and obtain the fatigue value level”; ¶ n0121 last 4 lines: “The analysis and processing module identifies the driver’s cold and hot” (determining respective set of input ambient conditions) “discomfort posture according to the driver’s posture” (based on the personalized rule) “issues instructions to the cab air-conditioning device” “to adjust the air volume, air temperature, wind speed, etc” (corresponding one or more additional actions are performed when similar ambient conditions are encountered)).
YANG BIN et al. do not specifically disclose:
wherein the natural language-based communication interface issues a query to the operator, wherein the query includes a structured prompt to the operator to provide the indication of the operator’s current fatigue level, wherein the indication is provided using natural language based on the query, wherein the natural language-based communication interface converts the query into the natural language.
XIAO GONGTING do teach: wherein the natural language-based communication interface issues a query to the operator, wherein the query includes a structured prompt to the operator to provide the indication of the operator’s current fatigue level, wherein the indication is provided using natural language based on the query, wherein the natural language-based communication interface converts the query into the natural language (¶ 0049 lines 8+: “The voice questions” (e.g., a query in natural language to a “driver” (operator)) “such as” “where is the Great Wall?” “If the driver replies “what? At home!”” (the operator response) “the intelligent voice” “module”(using a natural language-based communication interface) “determines that the driver is not mentally clear and is in a state of deep fatigue” (indicates their current fatigue level); ¶ 0070 lines 5+: “The intelligent voice interaction module continues to propose” “Are you in a bad mood?” (e.g., structured prompts to the operator) “the driver replies” “I want to sleep, I’m very sleepy” (the operator provides an indication regarding their current fatigue level in response to the prompt)).
It would have therefore been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the methods pertaining to detection and alleviation of “driver” “fatigue” by adding direct “question” “reply” interaction with a “driver” to enquire their state of XIAO GONGTING into the general “fatigue driving monitoring” of YANG BIN et al. would enable the combined systems and their associated methods to perform in combination as they do separately and to further enable YANG BIN to “solve the technical problems of inaccurate detection results” as disclosed in XIAO GONGTING Abstract last 3 lines.
Regarding claim 3, YANG BIN do not specifically disclose the method of claim 1, wherein the query is further structured to at least one of prompt the operator to provide the indication of a cause of the operator’s current fatigue level or obtain, using the natural language based communication interface, the indication from the operator of the cause of the operator’s current fatigue level.
XIAO GONGTING do teach the method of claim 1, wherein the query is further structured to at least one of prompt the operator to provide the indication of a cause of the operator’s current fatigue level or obtain, using the natural language based communication interface, the indication from the operator of the cause of the operator’s current fatigue level (¶ 0049 lines 8+: “The voice questions” (due to the prompting to the “driver” (operator)) “such as” “where is the Great Wall?” “If the driver replies “what? At home!”” (the operator response) “the intelligent voice interaction” “module” (via a natural language based communication interface) “determines that the driver is not mentally clear” (indicates a cause of the operator’s) “and is in a state of deep fatigue” (current fatigue level); ¶ 0070 lines 5+: “The intelligent voice interaction module continues to propose” “Are you in a bad mood?” (NLP interface prompts) “the driver replies” “I want to sleep, I’m very sleepy” (the operator provides being “sleepy” which is the cause of his “deep fatigue”)).
For obviousness to combine YANG BIN and XIAO GONGTING see claim 1.
Regarding claim 4, YANG BIN do not specifically disclose the method of claim 3, wherein the NLP unit is configured to process the obtained indications to determine a set of data points indicative of the operator's current fatigue level and also to determine a set of data points indicative of the cause of the operator’s current fatigue level.
XIAO GONGTING do teach the method of claim 3, wherein the NLP unit is configured to process the obtained indications to determine a set of data points indicative of the operator's current fatigue level and also to determine a set of data points indicative of the cause of the operator’s current fatigue level (¶ 0049 lines 8+: “The voice questions” “such as” “where is the Great Wall?” “If the driver replies “what? At home!”” “the intelligent voice” “module” (an NLP) “determines” (determines) “that the driver is not mentally clear” (data points indicative of a cause of the operator’s) “and is in a state of deep fatigue” (current fatigue level, where the “deep fatigue” is identified as “level 2” (a data point indicative of the operator’s current fatigue level; i.e., ¶ 0054 lines 1+: “fatigue driving is divided into 2 levels. Level 1 is when the driver is in primary fatigue driving, for example, the response to some questions is slow” “level 2” (a datapoint) “is when the driver is in deep fatigue” (corresponding to the indication) “driving, for example, the response to most questions are unreasonable” (indicative of strength as well as the cause of the operator’s current fatigue level)).
For obviousness to combine YANG BIN and XIAO GONGTING see claim 1.
Regarding claim 6, YANG BIN et al. in view of XIAO GONGTING do not specifically disclose the method of claim 1, further comprising:
after determining a set of one or more additional actions to be performed and the control circuit performing the set of one or more additional actions: the natural language-based communication interface issuing the query to the operator, wherein the query includes a structured prompt to the operator for validating the respective rule.
XIAO GONGTING do teach:
after determining a set of one or more additional actions to be performed and the control circuit performing the set of one or more additional actions: the natural language-based communication interface issuing the query to the operator, wherein the query includes a structured prompt to the operator for validating the respective rule
(¶ 0073 lines 13-14: “The intelligent voice interaction module determines that the driver is in a state of fatigue” (in response to a fatigue level detected) “The entertainment host will issue a voice reminder every 15 minutes” “You are a little tired now, please drive safely” (a set of actions to be performed are determined by the “intelligent voice interaction module” (functioning as both a control circuit as well as a natural language-based communication interface) which comprise of queries directed at the “driver” which amount to a rule being enforced for the driver’s safety once every 15 minutes, which for all the recitations after the first one amount to validation of the rule).
For obviousness to combine YANG BIN and XIAO GONGTING see claim 1.
Regarding claim 7, YANG BIN et al. do teach the method of claim 1, further comprising:
monitoring the one or more current ambient conditions within the system, and when the respective rule indicates that the control circuit should apply the one or more additional actions based on the one or more current ambient conditions, the control circuit then performing the corresponding one or more additional actions (¶ n0120 lines 2+: “temperature will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” “When the monitored air temperature is higher than 35 [degrees] C” (monitoring an ambient condition within the system subject to a respective rule) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on” (to perform a corresponding action) “the air conditioning device at the set temperature of 24 [degrees] C”).
Regarding claim 9, Yang Bin et al. do teach the method of claim 1, wherein the system is a collaborative human-robot system, wherein the control circuit is configured to control a robot that is performing one or more tasks in collaboration with a human operator, and wherein the one or more additional actions to be performed by the control circuit of the system to reduce the operator’s current fatigue level include at least one of:
reducing a speed of the robot:
controlling a heater to adjust an ambient temperature (¶ n0120 lines 2+: “temperature will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases”(in order to reduce the operator fatigue)“When the monitored air temperature is higher than 35 [degrees] C” “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device” (to a robot aiding a human “driver” (operator human passenger)) “to turn on the air conditioning device at the set temperature of 24 [degrees] C” (adjusts the ambient temperature (a condition within) of the vehicle));
or (iii) controlling a light source to adjust one or more ambient lighting conditions.
Regarding claim 10, YANG BIN et al. do teach the method of claim 1, wherein the system is a semi-autonomous or intelligent vehicle system (¶ n0119 describes a “cab” with “an intelligent voice module” (an intelligent vehicle system)),
wherein the control circuit is configured either to control a vehicle itself or to control one or more conditions within the vehicle, wherein the one or more additional actions to be performed by the control circuit of the system to reduce the operator’s current fatigue level include at least one of:
delegating control to an autonomous or advanced vehicle operator assistance module of the vehicle;
controlling a heater to adjust an ambient temperature within the vehicle (¶ n0120 lines 2+: “temperature will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases”(in order to reduce the operator fatigue)“When the monitored air temperature is higher than 35 [degrees] C” “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (adjusts the ambient temperature (a condition within) of the vehicle); or
controlling a light source to adjust ambient lighting conditions within the vehicle.
Regarding claim 11, YANG BIN et al. do teach a non-transitory computer readable storage medium storing software code that when executing on a data processor (¶ n0001: “The present invention belongs to the field of intelligent transportation technology, and in particular to a method and system for monitoring and reminding warning of fatigue driving based on computer vision technology”)
is configured to:
obtain, using a natural language-based communication interface, an indication from an operator of an operator’s current fatigue level (¶ n0114 lines 1+: “the monitoring system detects that the driver’s fatigue value is 3” (obtaining an indication of a “driver’s” (operator’s) current fatigue level) “the intelligent voice module responds” (using a natural language-based communication interface) “broadcasts the driver’s real-time fatigue value” (where the indication is obtained in natural language and indicates a current “fatigue value” (fatigue level) of the operator);
provide the indication to a natural language processing (NLP) unit, the NLP unit performing natural language-based processing on the indication to determine a set of one or more data points indicative of the operator’s current fatigue level (¶ n0114 lines 1+: “the monitoring system” (a natural language processing unit) “detects” (determines) “that the driver’s fatigue value is 3” (from the indication a data point “3” describing the fatigue level) “the intelligent voice module responds” “broadcasts the driver’s real-time fatigue value” “You are now severely fatigued”);
obtain, from one or more environmental sensors, data indicative of one or more current ambient conditions within a system, wherein the system is a collaborative system that is operated through a combination of one or more actions of the operator and one or more additional actions performed by a control circuit of the system, the one or more ambient conditions thereby being associated with the operator’s current fatigue level (¶ n0119 lines 3+: “A high concentration of CO2” (an ambient environmental condition) “increase their sleepiness” (associated with the operator’s fatigue level) “The CO2” “concentration monitoring device” (obtaining from one or more environmental sensors) “the CO2” “concentration in the cab” (data indicative of the current ambient conditions) “when the monitored CO2 concentration is higher than 1000 ppm, the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” “please open the window for ventilation” (i.e., a collaborative system which depends on actions from the operator and those mandated by a control circuit); ¶ n0120 lines 2+: “temperature” (another ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue) “The air temperature and humidity monitoring device” (a sensor) “monitors the temperature and humidity in the air in the cab” “When the monitored air temperature is higher than 35 [degrees] C” (obtains an environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (acts to adjust the temperature));
determine, using the determined set of one or more data points indicative of the operator’s current fatigue level and the associated one or more current ambient conditions, the one or more additional actions to be performed by the control circuit of the system to reduce operator fatigue (¶ n0119 lines 3+: “A high concentration of CO2” “increase their sleepiness” (fatigue measured by e.g., “fatigue value” (the one or more data points)) “The CO2” “concentration monitoring device” “monitors” “the CO2” “concentration in the cab” (and data indicative of the current ambient conditions) “When the monitored CO2 concentration is higher than 1000 ppm” “the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” (takes one or more actions) “please open the window for ventilation” (to help reduce operator fatigue); ¶ n0120 lines 2+: “temperature” (an ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue measured by “fatigue value” (the one or more data points)) “When the monitored air temperature is higher than 35 [degrees] C” (an obtained environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (determines and performs one or more actions to reduce operator’s fatigue level attributed to higher than 35 [degrees] C)); and
generate a respective rule personalized to the operator that can be automatically applied by the control circuit when similar ambient conditions are subsequently encountered within the system, the respective rule comprising an input set of ambient conditions and a corresponding one or more additional actions to be performed when the input set of ambient conditions is encountered (¶ n0006 last 3 lines: “the fatigue value real-time monitoring system is used to identify the fatigue characteristic posture of the driver” (an operator’s personalized rule generated) “and obtain the fatigue value level”; ¶ n0121 last 4 lines: “The analysis and processing module identifies the driver’s cold and hot” (determining respective set of ambient conditions) “discomfort posture according to the driver’s posture” (based on the personalized rule) “issues instructions to the cab air-conditioning device” “to adjust the air volume, air temperature, wind speed, etc” (corresponding one or more additional actions are performed)).
YANG BIN et al. do not specifically disclose: wherein the natural language-based communication interface issues a query to the operator, wherein the query includes a structured prompt to the operator to provide the indication of the operator’s current fatigue level, wherein the indication is provided using natural language based on the query, wherein the natural language-based communication interface converts the query into the natural language.
XIAO GONGTING do teach: wherein the natural language-based communication interface issues a query to the operator, wherein the query includes a structured prompt to the operator to provide the indication of the operator’s current fatigue level, wherein the indication is provided using natural language based on the query, wherein the natural language-based communication interface converts the query into the natural language (¶ 0049 lines 8+: “The voice questions” (e.g., a query in natural language to a “driver” (operator)) “such as” “where is the Great Wall?” “If the driver replies “what? At home!”” (the operator response) “the intelligent voice” “module” “determines that the driver is not mentally clear and is in a state of deep fatigue” (indicates their current fatigue level); ¶ 0070 lines 5+: “The intelligent voice interaction module continues to propose” “Are you in a bad mood?” (e.g., structured prompts to the operator) “the driver replies” “I want to sleep, I’m very sleepy” (the operator provides an indication regarding their current fatigue level in response to the prompt)).
It would have therefore been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the methods pertaining to detection and alleviation of “driver” “fatigue” by adding direct “question” “reply” interaction with a “driver” to enquire their state of XIAO GONGTING into the general “fatigue driving monitoring” of YANG BIN et al. would enable the combined systems and their associated methods to perform in combination as they do separately and to further enable YANG BIN to “solve the technical problems of inaccurate detection results” as disclosed in XIAO GONGTING Abstract last 3 lines.
Regarding claim 12, YANG BIN et al. do teach an apparatus for monitoring fatigue level of an operator (¶ n0001: “The present invention belongs to the field of intelligent transportation technology, and in particular to a method and system for monitoring and reminding warning of fatigue driving based on computer vision technology”)
The apparatus comprising:
a natural language-based communication interface operable to obtain from an operator of the system an indication in natural language of the operator’s current fatigue level (¶ n0114 lines 1+: “the monitoring system detects that the driver’s fatigue value is 3” (obtaining an indication of a “driver’s” (operator’s) current fatigue level) “the intelligent voice module responds” (using a natural language-based communication interface) “broadcasts the driver’s real-time fatigue value” (where the indication is obtained in natural language and indicates a current “fatigue value” (fatigue level) of the operator);
a natural language processing (NLP) unit operable to process the indication from the operator obtained via the natural language-based communication interface operable in order to determine a set of one or more data points indicative of the operator’s current fatigue level (¶ n0114 lines 1+: “the monitoring system” (a natural language processing unit) “detects” (determines) “that the driver’s fatigue value is 3” (from the indication a data point “3” describing the fatigue level) “the intelligent voice module responds” “broadcasts the driver’s real-time fatigue value” “You are now severely fatigued”);
a set of one or more environmental sensors operable to obtain data indicative of one or more current ambient conditions within the system, wherein the system is a collaborative system that is operated through a combination of one or more actions of the operator and one or more additional actions performed by a control circuit of the system (¶ n0119 lines 3+: “A high concentration of CO2” (an ambient environmental condition) “increase their sleepiness” (associated with the operator’s fatigue level) “The CO2” “concentration monitoring device” (obtaining from one or more environmental sensors) “the CO2” “concentration in the cab” (data indicative of the current ambient conditions) “when the monitored CO2 concentration is higher than 1000 ppm, the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” “please open the window for ventilation” (i.e., a collaborative system which depends on actions from the operator and those mandated by a control circuit); ¶ n0120 lines 2+: “temperature” (another ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue) “The air temperature and humidity monitoring device” (a sensor) “monitors the temperature and humidity in the air in the cab” “When the monitored air temperature is higher than 35 [degrees] C” (obtains an environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (acts to adjust the temperature));
and the control circuit for the system, wherein the control circuit is operable such that:
in response to the natural language-based communication interface obtaining the indication of the operator’s current fatigue level and the NLP unit then processing the indication to determine a set of one or more data points indicative of the operator’s current fatigue level: (¶ n0114 lines 1+: “the monitoring system” (a natural language processing unit) “detects” (determines) “that the driver’s fatigue value is 3” (from the indication a data point “3” describing the fatigue level) “the intelligent voice module responds” “broadcasts the driver’s real-time fatigue value” “You are now severely fatigued”):
the control circuit obtains the data indicative of the one or more current ambient conditions associated with the operator’s current fatigue level from the set of one or more environmental sensors (¶ n0119 lines 3+: “A high concentration of CO2” (an ambient environmental condition) “increase their sleepiness” (associated with the operator’s current fatigue level) “The CO2” “concentration monitoring device” (obtained from the one or more environmental sensors) “the CO2” “concentration in the cab” (data indicative of the current ambient conditions));
the control circuit then determines, using the determined set of one or more data points indicative of the operator’s current fatigue level and the associated one or more current ambient conditions, the one or more additional actions to be performed by the control circuit of the system to reduce operator fatigue (¶ n0119 lines 3+: “A high concentration of CO2” “increase their sleepiness” (fatigue measured by e.g., “fatigue value” (the one or more data points)) “The CO2” “concentration monitoring device” “monitors” “the CO2” “concentration in the cab” (and data indicative of the current ambient conditions) “When the monitored CO2 concentration is higher than 1000 ppm” “the analysis and processing module sends a signal to the intelligent voice module” (a control circuit of the system) “[which] advises the driver” (takes one or more actions) “please open the window for ventilation” (to help reduce operator fatigue); ¶ n0120 lines 2+: “temperature” (an ambient environmental factor) “will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases” (contributing to an operator’s fatigue measured by “fatigue value” (the one or more data points)) “When the monitored air temperature is higher than 35 [degrees] C” (an obtained environmental condition) “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (determines and performs one or more actions to reduce operator’s fatigue level attributed to higher than 35 [degrees] C)); and
the control circuit then generates a respective rule personalized to the operator that can be automatically applied by the control circuit when similar ambient conditions are subsequently encountered within the system, the respective rule comprising an input set of ambient conditions and a corresponding one or more additional actions to be performed when the input set of ambient conditions is encountered (¶ n0006 last 3 lines: “the fatigue value real-time monitoring system is used to identify the fatigue characteristic posture of the driver” (an operator’s personalized rule generated) “and obtain the fatigue value level”; ¶ n0121 last 4 lines: “The analysis and processing module identifies the driver’s cold and hot” (determining respective set of ambient conditions) “discomfort posture according to the driver’s posture” (based on the personalized rule) “issues instructions to the cab air-conditioning device” “to adjust the air volume, air temperature, wind speed, etc” (corresponding one or more additional actions are performed)).
YANG BIN et al. do not specifically disclose:
a natural language-based communication interface operable to issue a query and obtain from the operator of a system an indication in natural language of an operator’s current fatigue level.
XIAO GONGTING do teach: a natural language-based communication interface operable to issue a query and obtain from the operator of a system an indication in natural language of an operator’s current fatigue level (¶ 0049 lines 8+: “The voice questions” (e.g., a query in natural language to a “driver” (operator)) “such as” “where is the Great Wall?” “If the driver replies “what? At home!”” (the operator response) “the intelligent voice” “module” “determines that the driver is not mentally clear and is in a state of deep fatigue” (indicates their current fatigue level); ¶ 0070 lines 5+: “The intelligent voice interaction module continues to propose” “Are you in a bad mood?” (e.g., structured prompts to the operator) “the driver replies” “I want to sleep, I’m very sleepy” (the operator provides an indication regarding their current fatigue level in response to the prompt)).
It would have therefore been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the methods pertaining to detection and alleviation of “driver” “fatigue” by adding direct “question” “reply” interaction with a “driver” to enquire their state of XIAO GONGTING into the general “fatigue driving monitoring” of YANG BIN would enable the combined systems and their associated methods to perform in combination as they do separately and to further enable YANG BIN to “solve the technical problems of inaccurate detection results” as disclosed in XIAO GONGTING Abstract last 3 lines.
Regarding claim 15, YANG BIN et al. do teach the apparatus of any of claims 12, wherein the system comprises a collaborative human-robot system (¶ n0120 lines 2+: “temperature will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases”(in order to reduce the operator fatigue)“When the monitored air temperature is higher than 35 [degrees] C” “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device” (to a robot aiding a human “driver” (operator human passenger)) “to turn on the air conditioning device at the set temperature of 24 [degrees] C” (adjusts the ambient temperature (a condition within) of the vehicle))).
Regarding claim 16 YANG BIN et al. do teach the apparatus of claim 12, wherein the system comprises a semi-autonomous or intelligent vehicle system (¶ n0119 describes a “cab” with “an intelligent voice module” (an intelligent vehicle system)), ¶ n0120 lines 2+: “temperature will cause the driver to feel more uncomfortable, thereby accelerating the rate at which the driver’s fatigue level increases”(in order to reduce the operator fatigue)“When the monitored air temperature is higher than 35 [degrees] C” “and the relative humidity is higher than 80%” “the analysis and processing module” (the control circuit) “sends a signal to the cab air-conditioning device to turn on the air conditioning device at the set temperature of 24 [degrees] C” (adjusts the ambient temperature (a condition within) of the vehicle)).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang Bin et al. in view of XIAO GONGTING, and further in view of Breed et al. (US 2007/0025597).
Regarding claim 5, Yang Bin et al. in view of XIAO GONGTING do not specifically disclose the method of claim 1, wherein the NLP unit executes a machine learning model that has been trained to process the indication from the operator in order to determine at least a set of data points indicative of the operator’s current fatigue level or a cause of the operator’s current fatigue level.
Breed et al. do teach the method of claim 1, wherein the NLP unit executes a machine learning model that has been trained to process the indication from the operator in order to determine at least one of a set of data points indicative of the operator’s current fatigue level or a cause of the operator’s current fatigue level (¶ 0603 1st and last sentences respectively: “a vehicle interior monitoring system employing a sophisticated pattern recognition system, such as a neural network or modular neural network” (using a machine learning) “is in place, it is possible to monitor the motions of the driver over time and determine if he is falling asleep or has otherwise become incapacitated” “ the range of alert responses” (using a set of data points) “to the warning light and/or sound” (to process e.g. natural language of “driver” (operator)) “can be compared to the lack of response of a sleeping driver” (to determine his fatigue level) “and thereby the state of attentiveness determined”).
It would have therefore been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the “neural network” technique as used by Breed et al. in processing data associated with a “driver” to assess his “fatigue” to the corresponding techniques used by Yang Bin et al. in Yang Bin et al. in view of XIAO GONGTING would enable the combined systems and their associated methods to perform in combination as they do separately and enhance in processing “speed” for processing as disclosed in Breed et al. ¶ 0645 last sentence.
Claim(s) 8, 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang Bin et al. in view of XIAO GONGTING, and further in view of Kaushik et al. (US 2020/0317024).
Regarding claim 8, Yang Bin et al. in view of XIAO GONGTING do not specifically disclose the method of claim 1, wherein the respective rule is used as training data for training a model that is then executed by the control circuit in order to automatically adapt the system based