DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-15 are pending for examination.
Claims 1-15 are rejected under 35 U.S.C. §§ 112, 101.
Claims 1-2, 4-11 and 14-15 are rejected under 35 U.S.C. §102.
Claims 3 and 12-13 are rejected under 35 U.S.C. §103.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the weighted first and second latent space representations" in the last limitation. There is insufficient antecedent basis for this limitation in the claim.
Claim 5 recites the limitation "the determined characteristic" in the . There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites the limitation:
" the weighted first and second latent space representations " in "determining a characteristic of the user …" limitation,
“the determined characteristic” in “modifying audio or video content …” limitation, and
“the modified audio or video content” in “delivering the modified audio or video content …”.
There are insufficient antecedent bases for these limitations in the claim.
Claim 11 recites the limitation:
"the first, second and third soft latent space representations" in "calculate correlations between …" limitation,
“the correlations” in “weight each of …” limitation, and
“the characteristic” and “the weighted first, second and third latent space representations” in “determining the characteristic of a user …”.
There are insufficient antecedent bases for these limitations in the claim.
Claim 12 recites the limitation "the correlations" in the . There is insufficient antecedent basis for this limitation in the claim.
Claim 13 recites the limitation "the scaled correlation matrix" in the . There is insufficient antecedent basis for this limitation in the claim.
Claim 15 recites the limitation "weighted first, second and third latent space representations" and “the characteristic” and “the further weighted first, second, and third latent space representations” in the . There are insufficient antecedent bases for these limitations in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a Judicial Exception without significantly more.
Independent Claims
As Claims 1:
Step 1: Are the Claims to a process, machine, manufacture or composition of matter? Yes.
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. See the analysis below.
The Claim recites:
A system comprising:
a plurality of biometric sensors;
a first classifier engine to produce a first latent space representation of a first signal from a first biometric sensor of the plurality of biometric sensors;
a second classifier engine to produce a second latent space representation of a second signal from a second biometric sensor of the plurality of biometric sensors;
an attention engine to weight the first latent space representation and the second latent space representation based on correlation among latent space representations; and
a final classifier engine to determine a characteristic of a user based on the weighted first and second latent space representations.
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Regarding the non-emphasized limitations:
Step 2A prong 1:
“a first classifier engine to produce a first latent space representation of a first signal from a first biometric sensor of the plurality of biometric sensors;
a second classifier engine to produce a second latent space representation of a second signal from a second biometric sensor of the plurality of biometric sensors;
an attention engine to weight the first latent space representation and the second latent space representation based on correlation among latent space representations; and
a final classifier engine to determine a characteristic of a user based on the weighted first and second latent space representations.” is/are directed to a mental processes group of abstract idea. Mental processes are defined as concepts that can practically be performed in the human mind, or by a human using pen and paper as a physical aid. Examples of mental processes includes observations, evaluations, judgements and opinions.
These steps are considered mental processes group of abstract idea.
Step 2A prong 2:
Limitations “a plurality of biometric sensors; ” are insignificant extra solution activity. See MPEP §2106.05(g).
The Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that integrate the Judicial Exception into a practical application? No.
Limitation “a plurality of biometric sensors;” was considered insignificant extra solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)).
The claim is directed to mental processes group of abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
As Claims 6:
Step 1: Are the Claims to a process, machine, manufacture or composition of matter? Yes.
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. See the analysis below.
The Claim recites:
A method, comprising:
measuring a first biometric signal and a second biometric signal from a user of a head-mounted display;
generating a first latent space representation based on the first biometric signal;
generating a second latent space representation based on the second biometric signal;
weighting the first latent space representation and the second latent space representation based on correlations among latent space representations;
determining a characteristic of the user based on the weighted first and second latent space representations;
modifying audio or video content based on the determined characteristic; and
delivering the modified audio or video content to the user of the head- mounted display.
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Regarding the non-emphasized limitations:
Step 2A prong 1:
Limitations “generating a first latent space representation based on the first biometric signal;
generating a second latent space representation based on the second biometric signal;
weighting the first latent space representation and the second latent space representation based on correlations among latent space representations;
determining a characteristic of the user based on the weighted first and second latent space representations;” is/are directed to a mental processes group of abstract idea. Mental processes are defined as concepts that can practically be performed in the human mind, or by a human using pen and paper as a physical aid. Examples of mental processes includes observations, evaluations, judgements and opinions.
Limitation “modifying audio or video content based on the determined characteristic” is directed to a mathematical concepts group of abstract ideas. Mathematical concepts are defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations.
These steps are considered mental processes group of abstract idea.
Step 2A prong 2:
Limitations “
measuring a first biometric signal and a second biometric signal from a user of a head-mounted display;
delivering the modified audio or video content to the user of the head- mounted display.” are insignificant extra solution activity. See MPEP §2106.05(g).
The Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that integrate the Judicial Exception into a practical application? No.
Limitation “measuring a first biometric signal and a second biometric signal from a user of a head-mounted display;
delivering the modified audio or video content to the user of the head- mounted display” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.)
The claim is directed to mental processes group of abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
As Claims 11:
Step 1: Are the Claims to a process, machine, manufacture or composition of matter? Yes.
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. See the analysis below.
The Claim recites:
A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to:
generate a first latent space representation indicative of a characteristic of a user based on a first signal from a first biometric sensor;
generate a second latent space representation indicative of the characteristic of a user based on a second signal from a second biometric sensor;
generate a third latent space representation indicative of the characteristic of a user based on a third signal from a third biometric sensor;
calculate correlations between the first, second, and third soft latent space representations;
weight each of the first, second, and third latent space representations based on the correlations of that latent space representation with the other latent space representations; and
determine the characteristic of a user based on the weighted first, second, and third latent space representations.
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Regarding the non-emphasized limitations:
Step 2A prong 1:
Limitations “generate a first latent space representation indicative of a characteristic of a user based on a first signal from a first biometric sensor;
generate a second latent space representation indicative of the characteristic of a user based on a second signal from a second biometric sensor;
generate a third latent space representation indicative of the characteristic of a user based on a third signal from a third biometric sensor;
weight each of the first, second, and third latent space representations based on the correlations of that latent space representation with the other latent space representations; and
determine the characteristic of a user based on the weighted first, second, and third latent space representations.” is/are directed to a mental processes group of abstract idea. Mental processes are defined as concepts that can practically be performed in the human mind, or by a human using pen and paper as a physical aid. Examples of mental processes includes observations, evaluations, judgements and opinions.
Limitation “calculate correlations between the first, second, and third soft latent space representations;” is directed to a mathematical concepts group of abstract ideas. Mathematical concepts are defined as mathematical relationships, mathematical formulas or equations, or mathematical calculations.
These steps are considered mental processes group of abstract idea.
Step 2A prong 2:
There are no additional limitation(s).
The Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that integrate the Judicial Exception into a practical application? No. The claim is directed to mental processes group of abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Dependent Claims
As Claim 2, the Claim recites “wherein the attention engine is to apply a first weight to the first latent space representation, the first weight larger than a second weight applied to the second latent space representation, based on the first latent space representation being more highly correlated to other latent space representations than the second latent space representation.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the attention engine is to apply a first weight to the first latent space representation, the first weight larger than a second weight applied to the second latent space representation, based on the first latent space representation being more highly correlated to other latent space representations than the second latent space representation” is directed to mental processes group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 3, the Claim recites “further comprising a pre-processing engine to convert the first signal to a first time series, and a feature extraction engine to determine a first feature vector based on the first time series.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “further comprising a pre-processing engine to convert the first signal to a first time series, and a feature extraction engine to determine a first feature vector based on the first time series” is directed to mathematical calculations group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 4, the Claim recites “further comprising a third classifier engine to concatenate a third feature vector from a third biometric sensor with a fourth feature vector from a fourth biometric sensor and to produce a third latent space representation based on the concatenation of the third feature vector and the fourth feature vector, wherein the attention engine is to weight the third latent space representation, and wherein the final classifier engine is to determine the characteristic based on the weighted third latent space representation.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “further comprising a third classifier engine to concatenate a third feature vector from a third biometric sensor with a fourth feature vector from a fourth biometric sensor and to produce a third latent space representation based on the concatenation of the third feature vector and the fourth feature vector, wherein the attention engine is to weight the third latent space representation, and wherein the final classifier engine is to determine the characteristic based on the weighted third latent space representation” is directed to mental processes group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 5, the Claim recites “further comprising a head-mounted display, wherein the system is to alter an audio or video output by the head-mounted display based on the determined characteristic of the user.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the system is to alter an audio or video output by the head-mounted display based on the determined characteristic of the user” is directed to mathematical calculations group of abstract idea. Prong 2: The limitation “further comprising a head-mounted display” are insignificant extra solution activity. See MPEP §2106.05(g). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. The Limitations “comprising a head-mounted display” were considered insignificant extra solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). The Claim is not patent eligible.
As Claim 7, the Claim recites “further comprising training a first classifier to determine the characteristic based on the first biometric signal and training a second classifier to determine the characteristic based on the second biometric signal, wherein generating the first latent space representation comprises generating the first latent space representation using the first classifier, and wherein generating the second latent space representation comprises generating the second latent space representation using the second classifier.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein generating the first latent space representation comprises generating the first latent space representation using the first classifier, and wherein generating the second latent space representation comprises generating the second latent space representation using the second classifier” is directed to mathematical calculations group of abstract idea. Prong 2: The limitation “further comprising training a first classifier to determine the characteristic based on the first biometric signal and training a second classifier to determine the characteristic based on the second biometric signal,” are insignificant extra solution activity. See MPEP §2106.05(g). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. The Limitations “further comprising training a first classifier to determine the characteristic based on the first biometric signal and training a second classifier to determine the characteristic based on the second biometric signal,” were considered insignificant extra solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). The Claim is not patent eligible.
As Claim 8, the Claim recites “wherein the first and second classifiers include softmax functions to produce the first and second latent space representations, and wherein the method further comprises computing the correlation between the first latent space representation and the second latent space representation.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the first and second classifiers include softmax functions to produce the first and second latent space representations, and wherein the method further comprises computing the correlation between the first latent space representation and the second latent space representation” is directed to mathematical calculations group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 9, the Claim recites “wherein determining the characteristic of the user includes determining a cognitive load of the user.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein determining the characteristic of the user includes determining a cognitive load of the user” is directed to mental processes group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 10, the Claim recites “wherein modifying the audio or video content comprises modifying the audio or video content to cause an increase or decrease in the cognitive load of the user toward a predetermined cognitive load.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein modifying the audio or video content comprises modifying the audio or video content to cause an increase or decrease in the cognitive load of the user toward a predetermined cognitive load” is directed to mathematical concepts group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 12, the Claim recites “wherein the instructions to calculate the correlations include instructions that cause the processor to stack the first, second, and third latent space representations to form a matrix, multiply the matrix by its transpose to produce a correlation matrix, and scale the correlation matrix to produce a scaled correlation matrix.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the instructions to calculate the correlations include instructions that cause the processor to stack the first, second, and third latent space representations to form a matrix, multiply the matrix by its transpose to produce a correlation matrix, and scale the correlation matrix to produce a scaled correlation matrix” is directed to mathematical concepts group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 13, the Claim recites “wherein the instructions to weight each of the first, second, and third latent space representations include instructions that cause the processor to multiply the scaled correlation matrix with the matrix.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the instructions to weight each of the first, second, and third latent space representations include instructions that cause the processor to multiply the scaled correlation matrix with the matrix” is directed to mathematical concepts group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 14, the Claim recites “wherein the first latent space representation is a soft determination calculated by a classifier based on the first signal.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “wherein the first latent space representation is a soft determination calculated by a classifier based on the first signal” is directed to mathematical concepts group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
As Claim 15, the Claim recites “further comprising instructions to further weight each of the weighted first, second, and third latent space representations based on values of that representation, wherein the instructions to determine the characteristic comprise instructions that cause the processor to determine the characteristic based on the further weighted first, second, and third latent space representations.”
The non-emphasized limitations describe abstract processes while emphasized limitations recited additional limitation(s).
Step 2A: Are the Claims directed to a law of nature, a natural phenomenon (product of nature) or an abstract idea? Yes, the Claims is an abstract idea. Prong 1: The limitation “further comprising instructions to further weight each of the weighted first, second, and third latent space representations based on values of that representation, wherein the instructions to determine the characteristic comprise instructions that cause the processor to determine the characteristic based on the further weighted first, second, and third latent space representations” is directed to mental processes group of abstract idea. Prong 2: There are no additional limitation(s). Claim(s) does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: Does the Claim recite additional elements that amount to significantly more than the Judicial Exception? No. There are no additional limitation(s). The Claim is not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-11 and 14-15 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Seo et al. (U.S. 2020/0104670 hereinafter Seo).
As Claim 1, Seo teaches a system comprising:
a plurality of biometric sensors (Seo (¶0170 line 1-5), biometric sensors);
a first classifier engine to produce a first latent space representation of a first signal from a first biometric sensor of the plurality of biometric sensors (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
a second classifier engine to produce a second latent space representation of a second signal from a second biometric sensor of the plurality of biometric sensors (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
an attention engine to weight the first latent space representation and the second latent space representation based on correlation among latent space representations (Seo (¶0053 line 1-4, fig. 6), system applies predicted values to a weight model); and
a final classifier engine to determine a characteristic of a user based on the weighted first and second latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 2, besides Claim 1, Seo teaches wherein the attention engine is to apply a first weight to the first latent space representation, the first weight larger than a second weight applied to the second latent space representation, based on the first latent space representation being more highly correlated to other latent space representations than the second latent space representation (Seo (¶0107 line 5-9, fig. 6), different modality (latent space) weights differently based on emotion).
As Claim 4, besides Claim 1, Seo teaches further comprising a third classifier engine to concatenate a third feature vector from a third biometric sensor with a fourth feature vector from a fourth biometric sensor and to produce a third latent space representation based on the concatenation of the third feature vector and the fourth feature vector (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space)), wherein the attention engine is to weight the third latent space representation, and wherein the final classifier engine is to determine the characteristic based on the weighted third latent space representation (Seo (¶0053), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 5, besides Claim 1, Seo teaches further comprising a head-mounted display (Seo (¶0064 line 10-11), HMD (head-mounted display)), wherein the system is to alter an audio or video output by the head-mounted display based on the determined characteristic of the user (Seo (¶0145 last 3 lines), system provides output sound related to the predicted emotion).
As Claim 6, Seo teaches a method, comprising:
measuring a first biometric signal and a second biometric signal (Seo (¶0170 line 1-5), biometric sensors) from a user of a head-mounted display (Seo (¶0064 line 10-11), HMD (head-mounted display));
generating a first latent space representation based on the first biometric signal (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
generating a second latent space representation based on the second biometric signal (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
weighting the first latent space representation and the second latent space representation based on correlations among latent space representations (Seo (¶0053 line 1-4, fig. 6), system applies predicted values to a weight model);
determining a characteristic of the user based on the weighted first and second latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction);
modifyi ng audio or video content based on the determined characteristic (Seo (¶0145 last 3 lines), system provides output sound related to the predicted emotion); and
delivering the modified audio or video content to the user (Seo (¶0145 last 3 lines), system provides output sound related to the predicted emotion) of the head- mounted display (Seo (¶0064 line 10-11), HMD (head-mounted display)).
As Claim 7, besides Claim 6, Seo teaches further comprising training a first classifier (Seo (¶0177 last 6 lines), data learner trains neural network model to predict a person’s emotion) to determine the characteristic based on the first biometric signal and training a second classifier to determine the characteristic (Seo (¶0177 last 6 lines), data learner trains neural network model to predict a person’s emotion) based on the second biometric signal (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space)), wherein generating the first latent space representation comprises generating the first latent space representation using the first classifier, and wherein generating the second latent space representation comprises generating the second latent space representation using the second classifier (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space)).
As Claim 8, besides Claim 7, Seo teaches wherein the first and second classifiers include softmax functions to produce the first and second latent space representations (Seo (¶0095 last 3 lines), system operates based on argmax function), and wherein the method further comprises computing the correlation between the first latent space representation and the second latent space representation (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 9, besides Claim 6, Seo teaches wherein determining the characteristic of the user includes determining a cognitive load of the user (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 10, besides Claim 6, Seo teaches wherein modifying the audio or video content comprises modifying the audio or video content to cause an increase or decrease in the cognitive load of the user toward a predetermined cognitive load (Seo (¶0128 line 4-8, fig. 7), user feedback causes the emotion update from “Happy” to “Neutral”).
As Claim 11, Seo teaches a non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to:
generate a first latent space representation indicative of a characteristic of a user based on a first signal from a first biometric sensor (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
generate a second latent space representation indicative of the characteristic of a user based on a second signal from a second biometric sensor (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
generate a third latent space representation indicative of the characteristic of a user based on a third signal from a third biometric sensor (Seo (¶0088, ¶0089, fig. 4), system obtain plurality of biometric information. Each neural network models processes the collected data based on the data modality (a latent space));
calculate correlations between the first, second, and third soft latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction);
weight each of the first, second, and third latent space representations based on the correlations of that latent space representation with the other latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction); and
determine the characteristic of a user based on the weighted first, second, and third latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 14, besides Claim 11, Seo teaches wherein the first latent space representation is a soft determination calculated by a classifier based on the first signal (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
As Claim 15, besides Claim 11, Seo teaches further comprising instructions to further weight each of the weighted first, second, and third latent space representations based on values of that representation, wherein the instructions to determine the characteristic comprise instructions that cause the processor to determine the characteristic based on the further weighted first, second, and third latent space representations (Seo (¶0053, fig. 6), system applies predicted values to the weight model and produces an emotion prediction).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo in view of Lee et al. (U.S. 2022/0273210 hereinafter Lee).
As Claim 3, besides Claim 1, Seo does not explicitly disclose:
further comprising a pre-processing engine to convert the first signal to a first time series, and a feature extraction engine to determine a first feature vector based on the first time series.
Lee teaches:
further comprising a pre-processing engine to convert the first signal to a first time series, and a feature extraction engine to determine a first feature vector based on the first time series (Lee (¶0072, ¶0073), saccade onset time determines user’s region of interest. Saccade onset time can be a time series).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify sensor data of Seo instead be a saccade onset time taught by Lee, with a reasonable expectation of success. The motivation would be to conveniently allow the system to “distinguish and display a degree of interest and whether there is preference according to user’s gaze at the image content and provide them” (Lee (¶0125 last 3 lines)).
Claim(s) 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Seo in view of Janakiraman et al. (U.S. 2021/0287273 hereinafter Jana).
As Claim 12, besides Claim 1, Seo does not explicitly disclose:
wherein the instructions to calculate the correlations include instructions that cause the processor to stack the first, second, and third latent space representations to form a matrix, multiply the matrix by its transpose to produce a correlation matrix, and scale the correlation matrix to produce a scaled correlation matrix.
Jana teaches:
wherein the instructions to calculate the correlations include instructions that cause the processor to stack the first, second, and third latent space representations to form a matrix, multiply the matrix by its transpose to produce a correlation matrix, and scale the correlation matrix to produce a scaled correlation matrix (Jana (¶0035 last 5 lines), the recommendation engine calculate the score matrix by multiplying matrix A with a transpose of matrix A).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify weight matrix of Seo instead be a matrix taught by Jana, with a reasonable expectation of success. The motivation would be to conveniently allow that “the recommendation engine may efficiently calculate the score for each path using user/item matrix A” (Jana (¶0035 line 1-4)).
As Claim 13, besides Claim 1, Seo does not explicitly disclose:
wherein the instructions to weight each of the first, second, and third latent space representations include instructions that cause the processor to multiply the scaled correlation matrix with the matrix.
Jana teaches:
wherein the instructions to weight each of the first, second, and third latent space representations include instructions that cause the processor to multiply the scaled correlation matrix with the matrix (Jana (¶0035 last 5 lines), the recommendation engine calculate the score matrix by multiplying matrix A with a transpose of matrix A. The result is multiplied with the matrix A).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify weight matrix of Seo instead be a matrix taught by Jana, with a reasonable expectation of success. The motivation would be to conveniently allow that “the recommendation engine may efficiently calculate the score for each path using user/item matrix A” (Jana (¶0035 line 1-4)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Hussanmi et al. (U.S. 2020/0310540) teaches state prediction using neuromuscular data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NHAT HUY T NGUYEN whose telephone number is (571)270-7333. The examiner can normally be reached M-F: 12:00-8:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NHAT HUY T NGUYEN/Primary Examiner, Art Unit 2147