Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) limitations which under their broadest reasonable interpretation are directed to mental processes. This judicial exception is not integrated into a practical application as explained below. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception as explained below.
Regarding Claim 1, the claim recites a computer-implemented method for controlled a head mounted device, the computer-implemented method comprising:
Receiving, by an HMD controller, inertial measurement unit data generated in response to a vibration produced by a voiced nasal consonant produced by an HMD user
Analyzing, by the HMD controller, the IMU data to determine whether the IMU data corresponds to an HMD control command, and
Responsive to a determination that the IMU data corresponds to the HMD control command, sending an instruction corresponding with the HMD control command
Claim Interpretation: Under the broadest reasonable interpretation, the terms of the claim are
presumed to have their plain meaning consistent with the specification as it would be interpreted by
one of ordinary skill in the art. See MPEP 2111.
Receiving IMU data in response to a voice nasal consonant vibration by a user. A human can receive IMU data through seeing the data on a paper.
Analyzing the IMU data to determine whether the IMU data corresponds to a command. A human can analyze values to determine a match.
After determining a correspondence, sending an instruction corresponding with the command. A human can send an instruction verbally or through the use of pen and paper.
Additional elements: HMD controller and HMD control
Claim 1 is patent ineligible as it is directed to an abstract idea without being significantly more.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory
category. See MPEP 2106.03. The claim is directed to a method, which falls within one of the statutory
categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a
judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception
when the judicial exception is “set forth” or “described” in the claim.
As discussed above, the broadest reasonable interpretation of steps (a)-(c) is that those steps fall within the mental process groupings of abstract ideas because the cover concepts performed in the human mind, including observation, evaluation, judgement and opinion. See MPEP 2106.04(a)(2), subsection III.
Limitation a is directed to a mental step since a human can receive data. Limitation b is directed to a mental step since a human can determine if there is a match between the received data and a reference. Limitation c is directed to a mental step since a human can send an instruction based on the earlier determination. Hence these steps can be performed by a human using “observation, evaluation, judgement and opinion” because they involve making determinations and identifications,
which are mental tasks humans routinely do,' ” and thus can practically be performed in the human
mind, In re Killian, 45 F.4th 1373, 1379 (Fed. Cir. 2022). Therefore, these limitations are considered
together as an abstract idea for further analysis. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole
integrates the recited judicial exception into a practical application of the exception or whether the
claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether
there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating
those additional elements individually and in combination to determine whether the claim as a whole
integrates the exception into a practical application. See MPEP 2106.04(d). Earlier the additional elements were identified as the HMD controller and HMD control.
These additional elements provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. Even when viewed in combination the additional elements do not integrate the recited judicial exception into a practical application (Step 2A, Prong Two: NO), and the claim is directed to the judicial exception. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amount to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
At Step 2A, the additional elements of HMD controller and HMD control were found to represent no more than mere instruction to apply the judicial exception on a computer using generic computer components. Mere instructions to “apply” the abstract ideas, cannot provide an inventive concept. See MPEP 2106.05(f). The analysis under Step 2A, Prong Two is carried through to Step 2B. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept. (Step 2B: NO).
As such Claim 1 is patent ineligible.
The analysis above is applicable to Claims 15 and 21.
Regarding Claim 2, a human can identify features from a given IMU spectrogram and compare the identified features to a reference.
Analysis above is applicable to Claims 16 and 22 as well.
Regarding Claim 3, a human can hear the voiced nasal consonant. The audio sensor is an additional element but would not provide an inventive concept.
Regarding Claim 17 analysis analogous to Claim 3 is applicable.
Regarding Claim 4, the human mind can analyze if a sound corresponds to a command.
Regarding Claim 18, analysis analogous to Claim 4 is applicable.
Regarding Claim 5, the recited machine learning model would be mere instructions to apply the recited judicial exception as a human can recognize a sound like a voice nasal consonant.
This analysis is applicable to Claims 6, 19, 20 and 23.
Regarding Claim 7, a human can identify features from a given IMU spectrogram. Likewise, a human can classify the features and make determinations using them.
Analysis is applicable to Claim 8 as well.
Regarding Claim 9, a human can identify how confident one is at classifying.
Regarding Claim 10, a human can make a prediction
Regarding Claim 11, a human can combine given IMU data with observed eye-gaze information
Regarding Claim 12, a human can combine given IMU data with observed contact with the frame of an HMD.
Regarding Claim 13, a human can combine given IMU data with observed movement of a user.
Regarding Claim 14, a human can combine given IMU data with visual observations.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3-6, 11-23 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by LeBeau(US Pat 8,930,195).
Regarding Claim 1, LeBeau teaches a computer-implemented method for controlling a head mounted device (HMD), the computer-implemented method comprising: receiving, by an HMD controller, inertial measurement unit (IMU) data generated in response to a vibration produced by a voiced nasal consonant produced by an HMD user(causing at least one sensor of a head-mountable device to detect a vibration. The head-mountable device could include a near-eye display. The at least one sensor could be an accelerometer. The accelerometer could be operable to sense vibrations from a user of the head-mountable device. For example, the user may make nasal vocalization sounds and the accelerometer could be operable to transduce the sounds into electrical signals.[Col 13 Line 47-54], IMU(sensors, such as a proximity sensor and/or an inertial measurement unit (IMU)[Col 10 Line 33-34])); analyzing, by the HMD controller, the IMU data to determine whether the IMU data corresponds to an HMD control command(determining that the vibration corresponds to a first nasal vocalization from a plurality of nasal vocalizations[Col 13 Line 55-57], The first nasal vocalization could be determined based on the speech-recognition system comparing the vibration to a set of predetermined nasal vocalizations. The set of predetermined nasal vocalizations could include nasal vocalizations that could be used as command instructions for a user interface of the HMD[Col 3 Line 13-18]); and responsive to a determination that the IMU data corresponds to the HMD control command, sending an instruction corresponding with the HMD control command(Upon determining that the vibration corresponds to a first nasal vocalization, the HMD could control the user interface based on the first nasal vocalization. For instance, the user interface could carry out or dismiss various actions using the HMD based on the first nasal vocalization[Col3 Line 20-24], Upon determining the first nasal vocalization, a control instruction could be provided to the head-mountable device 402 based on the first nasal vocalization[Col 12 Line 64-66]).
Claim 15 recites similar limitations to Claim 1 and is rejected under the same rationale.
Claim 21 recites similar limitations to Claim 1 and is rejected under the same rationale.
Regarding Claim 3, LeBeau as in Claim 1 teaches receiving audio data from an audio sensor(microphone 338 [Figure 3]) connected to the HMD controller(causing at least one sensor of a head-mountable device to detect a vibration[Col 13 Line 47-48], transduce the sounds into electrical signals[Col 13 Line 53-54]), the audio data including the voiced nasal consonant corresponding to the IMU data(The accelerometer could be operable to sense vibrations from a user of the head-mountable device. For example, the user may make nasal vocalization sounds and the accelerometer could be operable to transduce the sounds into electrical signals.[Col 13 Line 50-54]).
Claim 17 recites similar limitations to Claim 3 and is rejected under the same rationale.
Regarding Claim 4, LeBeau as in Claim 3 teaches performing analysis of the audio data to confirm that the voiced nasal consonant corresponds to the HMD control command(determining that the vibration corresponds to a first nasal vocalization from a plurality of nasal vocalizations[Col 13 Line 55-57], The speech-recognition algorithms could use a database or datastore of speech audio files and/or text transcriptions in determining that the vibration corresponds to a first nasal vocalization.[Col 13 Line 63-66]).
Claim 18 recites similar limitations to Claim 4 and is rejected under the same rationale.
Regarding Claim 5, LeBeau as in Claim 3 teaches providing the audio data to a machine learning module executing an audio data ML model for recognition of the voiced nasal consonant(Such speech-recognition algorithms may include one or more of: a hidden Markov model, a dynamic time warping method, and neural networks. The speech-recognition algorithms could use a database or datastore of speech audio files and/or text transcriptions in determining that the vibration corresponds to a first nasal vocalization. The speech-recognition system could additionally or alternatively include a machine-learning algorithm.[Col 13-14 Lines 61-1]).
Claim 19 recites similar limitations to Claim 5 and is rejected under the same rationale.
Regarding Claim 6, LeBeau as in Claim 1 teaches providing the IMU data to a machine learning module executing an IMU data ML model to make the determination that the voiced nasal consonant corresponds to the HMD control command(Such speech-recognition algorithms may include one or more of: a hidden Markov model, a dynamic time warping method, and neural networks. The speech-recognition algorithms could use a database or datastore of speech audio files and/or text transcriptions in determining that the vibration corresponds to a first nasal vocalization. The speech-recognition system could additionally or alternatively include a machine-learning algorithm.[Col 13-14 Lines 61-1])..
Claim 20 recites similar limitations to Claim 6 and is rejected under the same rationale.
Claim 23 recites similar limitations to Claim 6 and is rejected under the same rationale.
Regarding Claim 11, LeBeau as in Claim 1 teaches combining the IMU data with predictive input data including user eye-gaze data(an eye-tracking system 302 may deliver eye-tracking data to the computer system 312 regarding the eye position of a wearer of the HMD 300. The eye-tracking data could be used, for instance, to determine a direction in which the HMD user may be gazing.[Col 9 Line 28-32]).
Regarding Claim 12, LeBeau as in Claim 1 teaches combining the IMU data with predictive input data(The finger-operable touch pad 124 may sense at least one of a position and a movement of a finger via capacitive sensing, resistance sensing, or a surface acoustic wave process, among other possibilities.[Col 5 Line 53-57]) comprising detection of a touch input on a frame of the HMD(The finger-operable touch pad 124 is shown on the extending side-arm 114 of the head-mountable device 102[Col 5 Line 47-48]).
Regarding Claim 13, LeBeau as in Claim 1 teaches combining the IMU data with predictive input data including a user movement(The location-sensing system 304 may include a gyroscope 320, a global positioning system (GPS) 322, and an accelerometer 324[Col 7 Line 63-65], The location-sensing system 304 could include other sensors, such as a proximity sensor and/or an inertial measurement unit (IMU)[Col 10 Line 32-34]).
Regarding Claim 14, LeBeau as in Claim 1 teaches combining the IMU data with predictive input data including image sensor data(The camera 340 could represent any device configured to capture an image, such as an image-capture device[Col 11 Line 19-21]).
Regarding Claim 16, LeBeau as in Claim 1 teaches machine-readable instructions further cause the processor to compare the IMU data with stored IMU data pre-recorded for the HMD user(Such speech-recognition algorithms may include one or more of: a hidden Markov model, a dynamic time warping method, and neural networks. The speech-recognition algorithms could use a database or datastore of speech audio files and/or text transcriptions in determining that the vibration corresponds to a first nasal vocalization. The speech-recognition system could additionally or alternatively include a machine-learning algorithm.[Col 13-14 Lines 61-1])..
Claim 22 recites similar limitations to Claim 16 and is rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over LeBeau(US Pat 8,930,195) in view of Stewart(US PGPub 20230213936).
Regarding Claim 2, LeBeau as in Claim 1 teaches analyzing the IMU data further comprising comparing IMU data with pre-recorded IMU data for the HMD user(configured to compare a vibration (which may be detected using a sensor, such as an accelerometer) to a set of predetermined nasal vocalizations[Col 11 Line 28-30]).
LeBeau does not teach feature embeddings from a IMU spectrogram with a class prototype of IMU feature embeddings, the class prototype of IMU feature embeddings being produced based on IMU data.
However, Stewart teaches feature embeddings from a IMU spectrogram with a class prototype of IMU feature embeddings, the class prototype of IMU feature embeddings being produced based on IMU data(to extract the features for the plurality of inertial measurement data sets comprises at least one of: performing a convolutional neural network; performing an autoencoder; converting the inertial measurement data sets into at least one of: a frequency-domain representation; time-domain embedding; and spectrogram[0065]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the system of LeBeau with the feature extraction method of Stewart because it would reduce the effects of noise and errors on IMU measurements[0012].
Claim(s) 7-10 are rejected under 35 U.S.C. 103 as being unpatentable over LeBeau(US Pat 8,930,195) in view of Stewart(US PGPub 20230213936) and further in view of Khan(US PGPub 20180197078).
Regarding Claim 7, LeBeau teaches classifying and determining the HMD control command(Upon determining the first nasal vocalization is an affirmative vocalization, the head-mountable device could be controlled to carry out the action. Upon determining the first nasal vocalization is a negative vocalization, the head-mountable device could be controlled to dismiss the action or carry out a different action.[Col 14 Line 20-26]).
LeBeau does not teach extracting feature embeddings from an IMU spectrogram representing the IMU data; classifying the feature embeddings from the IMU spectrogram using an IMU classifier as a class prototype of IMU feature embeddings.
However, Stewart teaches extracting feature embeddings from an IMU spectrogram representing the IMU data(to extract the features for the plurality of inertial measurement data sets comprises at least one of: performing a convolutional neural network; performing an autoencoder; converting the inertial measurement data sets into at least one of: a frequency-domain representation; time-domain embedding; and spectrogram[0065]);
Stewart does not teach classifying the feature embeddings from the IMU spectrogram using an IMU classifier as a class prototype of IMU feature embeddings.
However, Khan teaches classifying the feature embeddings from the IMU spectrogram using an IMU classifier as a class prototype of IMU feature embeddings(he thus extracted features are used as an input for a first classification step 170. This classification step produces, as its output, an indication of the type of activity represented by the data[0056]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the system of LeBeau with the feature extraction method of Stewart because it would reduce the effects of noise and errors on IMU measurements[0012]. Additionally, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the system of LeBeau with the feature classification of Khan because it would increase classifier performance[0076].
Regarding Claim 8, LeBeau in view of Stewart teaches the class prototype (predetermined nasal vocalizations[Col 11 Line 28-30]) of the IMU feature(extract the features for the plurality of inertial measurement data sets[0065]) embedding corresponds with the voiced nasal consonant.
Regarding Claim 9, Khan teaches classifying includes calculating a confidence level(as its output, an indication of the type of activity represented by the data as well as a confidence score associated with the classification step[0056]).
Regarding Claim 10, Khan teaches classifying includes calculating prediction data(as its output, an indication of the type of activity represented by the data as well as a confidence score associated with the classification step[0056]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARJUN R SWAMY whose telephone number is (571)272-9763. The examiner can normally be reached Mon, Tue, Thur, Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARJUN SWAMY/Examiner, Art Unit 2654
/Richa Sonifrank/Primary Examiner, Art Unit 2654
Prosecution Insights