Detailed Action
This communication is in response to the Arguments and Amendments filed on 2/15/2026.
Claims 1-3, 5-11, and 13-30 are pending and have been examined.
Claims 4 and 12 have been cancelled.
Independent Claims 1, and 22 are parallel device and method claims, respectively.
Apparent priority: 08/04/2021.
Any previous objection/rejection not mentioned in this Office Action has been withdrawn by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/2/2025, 12/4/2025, 3/1/2026, have been considered by the examiner.
Response to Amendment
The Applicant has amended the claims to include “cheek” “cheek” and “responsively to changes in the signal output by the optical sensing head due to movements of a skin surface of the user without any utterance of sounds by the user.”
As to the claim rejections 35 U.S.C. § 103 Applicant notes,
Applicant notes Liu teaches an image capturing device configured for "successively capturing images of lips 902 of the subject 900 during a speech of the subject 900." Liu, paragraph [0038]. Liu does not teach or suggest an optical sensing head configured to sense light reflected from a cheek of the user, as now recited by amended claim 1. Liu's system captures images of the lips, not reflected light from the cheek. Furthermore, Liu requires actual speech from the subject, as the system operates when "the subject 900, who is wearing the wearable accessory 3 and the oral cavity detecting device 5" speaks, and "during speech of the subject 900, the image capturing device 4 and the oral cavity detecting device 5 obtain the speech information." Liu, paragraphs [0075]-[0076]. Liu does not teach generating speech output from skin movements "without any utterance of sounds by the user" as now recited by amended claim 1.
Examiner notes Bakish has been added to the claim mapping. Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
Applicant notes Holzrichter teaches an EM sensor positioned to detect glottal motions and an acoustic sensor positioned to receive acoustic output from the mouth, where "acoustic pressure waves" are measured "by the acoustic sensor (e.g., a microphone)." HoIzrichter, Column 16, Lines 4-22. Holzrichter does not cure the deficiencies of Liu, as Holzrichter does not teach sensing light reflected from a cheek or generating speech output without any utterance of sounds. Accordingly, neither Liu nor Holzrichter, alone or in combination, teaches or suggests an optical sensing head configured to sense light reflected from a cheek of the face, combined with processing circuitry configured to generate speech output responsively to changes in the signal due to movements of a skin surface without any utterance of sounds by the user, as recited by amended claim 1.
Examiner notes this limitation is mapped to Bakish, not to Holzrichter or Liu. Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
In rejecting claim 4, the Examiner acknowledged that Liu and Holzrichter do not teach sensing light reflected from the cheek of a user and instead cited Bakish, who describes a laser microphone including a laser transmitter operating by self-mixing interferometry. Rakish, abstract. Bakish mentions in passing that the laser beam may be reflected from the cheek of a human speaker to detect, measure, or estimate skin vibrations. Bakish, paragraph [0049]. Bakish relates only to "spoken speech" in this and every other context and does not even hint that the detected vibrations of the cheek could possibly be used generate a speech output without any utterance of sounds by the user, as recited in amended claim 1.
Examiner notes Bakish teaches skin vibrations in [0049] which can be more than just spoken speech.
Applicant notes In rejecting claim 12, the Examiner acknowledged that Liu and Holzrichtcr do not teach generating a speech output without any utterance of sounds by the user and instead cited Zamora. Zamora teaches receiving depth information corresponding to a face, detecting lips, and tracking lip descriptor points to recognize speech based on a generated visual pattern. Zamora's techniques enable speech to be recognized using depth information, where "the depth information may be used to detect lip movements via lip descriptors." Zamora, Column 2, Lines 8-26. Zamora focuses on lip movements detected via depth cameras during actual speech by a person and makes no suggestion that his lip tracking technique could be used without utterance of sounds by the user, contrary to the limitations of claim 1.
Examiner notes that Bakish has been added to teach these portions of the claim mapping. Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
Applicant notes It appears that the Examiner may be implying that Zamora's method of tracking lips could be applied even when the subject is moving her lips without speaking. This interpretation of Zamora may or may not be correct, but Zamora himself says nothing about it. The only motivation to modify Zamora to operate in this manner is hindsight from the present patent application. Furthermore, Zamora is concerned only with detecting lip movements and says nothing at all about sensing fine movements of skin surfaces (such as the cheek), let alone movements that occur during silent articulation without any vocalization. Therefore, even if the person of ordinary skill were to have attempted to apply Zamora to the problem of deciphering speech without utterance of sounds, that person would have applied Zamora's8 teachings to Liu's lip sensing and would not even have considered using light reflected from the cheek for such a purpose.
Examiner notes that lips are a skin surface which make finer movements than cheeks, therefore the tracking that Zamora achieves could easily be applied to the less fine movements of the cheeks. Bakish has been mapped to the cheek. A person of ordinary skill in the art would consider it obvious to combine the Speech Recognition Using Depth Information of Zamora with the visual speech recognition of Bakish to recognize words from visual patterns.
Regarding the rejection of claims 3, 5-11, 13, and 20 over Liu in view of Holzrichter and further in view of Petersen, Petersen teaches a gaze detection arrangement comprising a laser device with surface emitters that irradiate laser beams onto the eye for determining a gaze direction. Petersen, paragraphs [0038]-[0039]. Petersen does not teach sensing light reflected from a cheek of the user or generating speech output from skin movements without any utterance of sounds. The combination of Liu, Holzrichter, and Petersen does not teach or suggest the limitations now recited by amended claim 1. Claims 3, 5-11, 13, and 20 are allowable for at least the same reasons as claim 1.
Examiner notes Bakish has been added to the claim mapping to teach the amended portions of the independent claims.
Regarding the rejection of claims 14 and 21 over Liu in view of Holzrichter and further in view of Oba, Oba teaches a cochlear implant system and an auditory brainstem implant system for presenting speech information to a user through clcctrodcs. Oba, Column 24, Lines 4-18. Oba does not teach sensing light reflected from a cheek of the user or generating speech output from skin movements without any utterance of sounds. The combination of Liu, Holzrichter, and Oba does not teach or suggest the limitations now recited by amended claim 1. Claims 14 and 21 are allowable for at least the same reasons as claim 1.
Examiner notes Bakish has been added to the claim mapping to teach amended claims 1, 14 and 21. Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
Regarding the rejection of independent claim 22 over Liu in view of Zamora, amended claim 22 recites a method for sensing comprising sensing a movement of skin on a cheek of a human subject in response to words articulated by the subject without vocalization of the words by the subject and without contacting the skin, and responsively to the sensed movement, generating a speech output including the articulated words.
Examiner notes Bakish has been added to the claim mapping to teach the cheek. Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
As explained above in regard to claim 1, Liu teaches capturing images of lips during speech and does not teach sensing a movement of skin on a face in response to words articulated without vocalization and without contacting the skin, as recited by claim 22.
Zamora teaches receiving depth information corresponding to a face, detecting lips, and tracking lip descriptor points to recognize speech based on a generated visual pattern. Zamora's techniques enable speech to be recognized using depth information, where "the depth information may be used to detect lip movements via lip descriptors." Zamora, Column 2, Lines 8-26. Zamora focuses on lip movements detected via depth cameras during actual speech by a person and makes no suggestion that his lip tracking technique could be used without utterance of sounds by the user, contrary to the limitations of claim 22.
Examiner notes Bakish has been added to the claim mapping.
Examiner notes that lips are a skin surface which make finer movements than cheeks, therefore the tracking that Zamora achieves could easily be applied to the less fine movements of the cheeks. Bakish has been mapped to the cheek. A person of ordinary skill in the art would consider it obvious to combine the Speech Recognition Using Depth Information of Zamora with the visual speech recognition of Bakish to recognize words from visual patterns.
Examiner notes that lips make finer movements than cheeks, therefore the tracking that Zamora achieves could easily be applied to the less fine movements of the cheeks.
As noted earlier, it appears that the Examiner may be implying that Zamora's method of tracking lips could be applied even when the subject is moving her lips without speaking. This interpretation of Zamora may or may not be correct, but Zamora himself says nothing about it. The only motivation to modify Zamora to operate in this manner is hindsight from the present patent application. Furthermore, Zamora is concerned only with detecting lip movements and says nothing at all about sensing fine movements of skin surfaces (such as the cheek), let alone movements that occur during silent articulation without any vocalization. Therefore, even if the person of ordinary skill were to have attempted to apply Zamora to the problem of deciphering speech without utterance of sound that person would have applied Zamora's teachings to Liu's lip sensing and would not even have considered using light reflected from the cheek for such a purpose.
Thus, the combination of Liu and Zamora does not teach or suggest "sensing a movement of skin on a cheek of a human subject in response to words articulated by the subject without vocalization of the words by the subject and without contacting the skin" as recited by claim 22. Therefore, claim 22 is patentable over the cited art.
Examiner notes Bakish has been added to the claim mapping.
Examiner further notes that Bakish teaches in [0049] an optical sensing head “the optical microphone or laser microphone” configured to sense light reflected from a cheek of the user, “measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear” generating speech output from skin movements without any utterance of sounds by the user see Bakish [0066] “dynamically generate and may dynamically apply, to the acoustic signal”
Examiner notes that lips are a skin surface which make finer movements than cheeks, therefore the tracking that Zamora achieves could easily be applied to the less fine movements of the cheeks. Bakish has been mapped to the cheek. A person of ordinary skill in the art would consider it obvious to combine the Speech Recognition Using Depth Information of Zamora with the visual speech recognition of Bakish to recognize words from visual patterns.
Regarding the rejection of claim 23 over Liu in view of Zamora and further in view of Bakish, Bakish teaches an optical microphone that "operates to remotely detect or measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, car)." Bakish, paragraph [0049]. However, Bakish detects vibrations caused by spoken speech, not movements of skin in response to words articulated without vocalization. The combination of Liu, Zamora, and Bakish does not teach or suggest sensing movement of skin on a face in response to words articulated without vocalization, as required by claim 22 from which claim 23 depends. Claim 23 is allowable for at least the same reasons as claim 22.
Examiner notes Bakish does teach this limitation [0053] System 1100 may comprise, for example, an optical microphone 1101 able to transmit an optical beam (e.g., a laser beam) towards a target (e.g., a face of a human speaker), and able to capture and analyze the optical feedback that is reflected from the target, particularly from vibrating regions or vibrating face-regions or face-portions of the human speaker. The optical microphone 1101 may be or may comprise or may utilize a Self-Mix (SM) chamber or unit, an interferometry chamber or unit, an interferometer, a vibrometer, a targeted vibrometer, or other suitable component, able to analyze the spectrum of the received optical signal with reference to the transmitted optical beam, and able to remotely estimate the audio or speech or utterances generated by the target (e.g., the human speaker).
Applicant notes Regarding the rejection of claims 24-28 over Liu in view of Zamora and further in view of Petersen, Petersen teaches laser feedback interferometry for determining an optical path length of an emitted laser beam to the surface of the eye. Petersen, paragraph [0011]. Petersen is directed to eye tracking, not to sensing skin movements on a face in response to silent articulation of words. The combination of Liu, Zamora, and Petersen does not teach or suggest the limitations of claim 22 from which claims 24-28 ultimately depend. Claims 24-28 are allowable for at least the same reasons as claim 22.
Examiner notes Bakish has been added to the claim mapping to teach the amended portions of claim 22, as noted above.
Regarding the rejection of claims 29 and 30 over Liu in view of Zamora and further in view of Begum, Begum teaches that similar facial features produce similarly sounding voices and uses correlation between facial features and sound frequencies to synthesize a voice based on facial structure. Begum, paragraphs [0013]-[0014]. Begum is directed to text-to-speech synthesis based on facial images, not to sensing skin movements in response to silent articulation. The combination of Liu, Zamora, and Begum does not teach or suggest the limitations of claim 22 from which claims 29 and 30 depend. Claims 29 and 30 are allowable for at least the same reasons as claim 22.
Examiner notes Bakish has been added to the claim mapping to teach the amended portions of claim 22, as noted above.
Applicant's arguments have been fully considered but they are not persuasive.
Updated mappings to reflect the amendments to the independent claims have been provided below in view of Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of in view of Holzrichter (U.S. Patent Number US 5729694 A).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of in view of Holzrichter (U.S. Patent Number US 5729694 A),
Regarding independent Claim 1, Liu teaches
an optical sensing head held by the bracket in a location in proximity to a cheek of the user and configured to sense light (see Liu [0047] In use, each of the distance sensors 531 is able to send a probing signal (e.g., a light signal) toward the tongue 905, and to output a distance detecting signal according to the probing signal reflected by the tongue 905. Using the distance detecting signals outputted by the distance sensors 531, the signal processor 532 is then able to determine distances between the respective distance sensors 531 and the tongue 905 during the speech of the subject 900, and generate the distance signal accordingly. Similarly, the distance signal can indicate a change pattern of the distance between the palate 904 and the tongue 905 during the speech since the carrier base 51 provided with the distance detecting unit 53 serves as the replacement of the palate 904.”) (examiner notes the tongue is in proximity to the cheek) and to output a signal in response to the detected light; and (see Liu Abstract “A speech recognition system is to be used on a human subject. The speech recognition system includes an image capturing device, an oral cavity detecting device and a speech recognition device. The image capturing device captures images of lips of the subject during a speech of the subject. The oral cavity detecting device detects contact with a tongue of the subject and distance from the tongue of the subject, and accordingly generates a contact signal and a distance signal. The speech recognition device processes the images of the lips and the contact and distance signals so as to obtain content of the speech of the subject.”) processing circuitry configured to process the signal to generate a speech output. (see Liu [0068] “The output device 7 includes an audio output module 71 for outputting the synthesized pronunciation signal, and a display module 72 for displaying a word according to the word data sets received from the speech recognition device 6.”)
Liu does not specifically teach reflected from the cheek However, Bakish does teach this limitation. (see Bakish [0049] “In some embodiments which may utilize a laser microphone or optical microphone, such optical microphone (or optical sensor) and/or its components may be implemented as (or may comprise) a Self-Mix module; for example, utilizing a self-mixing interferometry measurement technique (or feedback interferometry, or induced-modulation interferometry, or backscatter modulation interferometry), in which a laser beam is reflected from an object, back into the laser. The reflected light interferes with the light generated inside the laser, and this causes changes in the optical and/or electrical properties of the laser. Information about the target object and the laser itself may be obtained by analyzing these changes. In some embodiments, the optical microphone or laser microphone operates to remotely detect or measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear); and/or to remotely detect or measure or estimate the direct changes in skin vibrations; rather than trying to measure indirectly an effect of spoken speech on a vapor that is exhaled by the mouth of the speaker, and rather than trying to measure indirectly an effect of spoken speech on the humidity or relative humidity or gas components or liquid components that may be produced by the mouth due to spoken speech.”) responsively to changes in the signal output by the optical sensing head due to movements of a skin surface of the user without any utterance of sounds by the user. (see Bakish [0031] The Applicants have realized that this may entail disadvantages, since the actual location of the speaker (or the speaker's mouth or face) may not be exactly at the estimated target location, or since the speaker may move (e.g., performing slight or natural movements of the face) while speaking; thereby generating noise and/or otherwise reducing the accuracy or efficiency of the laser-based microphone. The Applicants have devised improved system(s) able to mitigate or eliminate such disadvantages. [0032] In a first demonstrative implementation of the present invention, a matrix or array of multiple, discrete, laser beams (or laser rays) may be utilized, to perform multiple laser-based readings of multiple nearby locations in the target area. The resulting signals may be fused or combined, or a “best-of” signal may be selected from each set of captured signals, in order to produce the best-available optical reading(s) at each time-point or time-slot.”)(see Bakish [0066] Optionally, system 1100 may comprise a signal enhancer 1109, which may enhance, filter, improve and/or clean the acoustic signal that is captured by acoustic microphone 1102, based on output generated by the optical microphone 1101. For example, system 1100 may dynamically generate and may dynamically apply, to the acoustic signal captured by the acoustic microphone 1102”)
Liu and Bakish are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method combination of Liu to include the on a cheek of the user and responsively to changes in the signal output by the optical sensing head due to movements of a skin surface of the user without any utterance of sounds by the user of Bakish. This allows selection of optimal signals as recognized by Bakish [0036].
Liu does not specifically teach 1. (Original) A sensing device, comprising: a bracket configured to fit an ear of a user of the device; However, Holzrichter does teach this limitation (see Holzrichter (56:51-55) “Unit 99 and speaker 100 are mounted in an ear piece 101 of handset 90 so that the speaker 100 is positioned over the person's ear. Several system functions illustrated in FIG. 20 are similar to those shown in FIG. 8.”)
Liu in view of Bakish and Holzrichter are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device of combination of Liu and Bakish to incorporate A sensing device, comprising: a bracket configured to fit an ear of a user of the device of Holzrichter. This allows optimized positioning of the device as recognized by Holzrichter (56:25-55).
As to Claim 2, Liu in view of Bakish and further in view of Holzrichter teaches 2. (Original) The device according to claim 1,
Furthermore, Holzrichter teaches wherein the bracket comprises an ear clip. (see Holzrichter (56:51-55) “Unit 99 and speaker 100 are mounted in an ear piece 101 of handset 90 so that the speaker 100 is positioned over the person's ear. Several system functions illustrated in FIG. 20 are similar to those shown in FIG. 8.”)
Liu in view of Bakish and further in view of Holzrichter are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include the bracket comprises an ear clip of Holzrichter. This allows optimized positioning of the device as recognized by Holzrichter (56:25-55).
As to Claim 15, Liu in view of Bakish and further in view of Holzrichter teaches 15. (Currently amended) The device according to claim 1,
Furthermore, Liu teaches and comprising a microphone configured to sense sounds uttered by the user. (see Liu [0069] “As shown in FIG. 2, in this embodiment, the audio output module 71 is embodied using a microphone disposed on the free end of the extension part 33, and is coupled to the speech recognition device 6 via the cable wire in the extension part 33 for receiving the synthesized pronunciation signal from the speech recognition device 6.”)
As to Claim 16, Liu in view of Bakish and further in view of Holzrichter teaches 16. (Currently amended) The device according to claim 15,
Furthermore, Holzrichter teaches wherein the processing circuitry is configured to compare the signal output by the optical sensing head to the sounds sensed by the microphone in order to calibrate the optical sensing head. (see Holzrichter (7:5-20) “(35) For each set of received EM signals and acoustic signals there is a need to process and extract the information on organ positions (or motions) and to use the coded speech sounds for the purposes of deconvolving the excitation from the acoustic output, and for tract configuration identification. For example, information on the positions of the vocal folds (and therefore the open area for air flow) vs. time is obtained by measuring the reflected EM waves as a function of time. Similarly, information on the conditions of the lips, jaw, teeth, tongue, and vellum positions can be obtained by transmitting EM waves from other directions and using other pulse formats. The reflected and received signals from the speech organs are stored in a memory and processed every speech time frame, as defined below. The reflected EM signals can be digitized, averaged, and normalized, as a function of time, and feature vectors can be formed.”) (see Holzrichter 8:21-40) “(39) Secondly, a synthesized output acoustic function, I(t), can be produced by convolving the voiced excitation function, E(t), with the transfer function, H(t), for each desired acoustic speech unit. Thirdly, the excitation function, E, can be determined by deconvolving a previously obtained transfer function, H, from a measured acoustic output function, I. This third method is useful to obtain the modified-white-noise excitation-source spectra to define an excitation function for each type of unvoiced excitation. In addition, these methods can make use of partial knowledge of the functional forms E, H, or I for purposes of increasing the accuracy or speed of operation of the processing steps. For example, the transfer function H is known to contain a term R which describes the lips-to-listener free space acoustic radiation transfer function. This function R can be removed from H leaving a simpler function, H*, which is easier to normalize. Similar knowledge, based on known acoustic physics, and known physiological and mechanical properties of the vocal organs, can be used to constrain or assist in the coding and in specific applications.”)
Liu in view of Bakish and further in view of Holzrichter are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and of Bakish and of Holzrichter to include the processing circuitry is configured to compare the signal output by the optical sensing head to the sounds sensed by the microphone in order to calibrate the optical sensing head of Holzrichter. This allows or purposes of increasing the accuracy or speed of operation as recognized by Holzrichter (8:31-32).
As to Claim 17, Liu in view of Bakish and further in view of Holzrichter teaches 17. (Currently amended) The device according to claim 15,
Furthermore, Liu teaches wherein the processing circuitry is configured to change an operational state of the device responsively to sensing of the sounds uttered by the user. (see Liu [0077] “In the case that no speech parameter set stored in the database 631 corresponds to the content of the speech, the output device 7 may be controlled by the speech recognition unit 63 to output a notification to the subject 900, and to instruct him/her to speak again for recognization.”)
As to Claim 18, Liu in view of Holzrichter teaches 18. (Currently amended) The device according to claim 1,
Furthermore, Liu teaches and comprising a communication interface, wherein the processing circuitry is configured to encode the signal for transmission over the communication interface to a processing device, which processes the encoded signals to generate the speech output. (see Liu [0086] “In such a case, the components that are for processing signals, such as the signal processors 522 and 532, may be disposed in the wearable accessory 3 instead of the carrier base 51. The signals generated by the pressure sensors 521 (only one is depicted in FIG. 9) and the distance sensors 531 (only one is depicted in FIG. 9) are transmitted via the first communication unit 55 to the signal processors 522 and 532 for processing. As a result, the size of the carrier base 51 may be made smaller due to the reduced number of the components incorporated therein.”)
As to Claim 19, Liu in view of Bakish and further in view of Holzrichter teaches 19. (Currently amended) The device according to claim 18,
Furthermore, Liu teaches wherein the communication interface comprises a wireless interface. (see Liu [0049] “The first communication unit 55 is coupled to the speech recognition device 6 wirelessly, in order to transmit the signals generated by the contact detecting unit 52, the distance detecting unit 53, and the airflow detecting unit 54 to the speech recognition device 6. Data transmission between the first communication unit 55 and the speech recognition device 6 may be performed using the ANT+ protocol, Bluetooth®, ZigBee®, near field communication (NFC), etc.”)
Claims 3, 5-11, 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of Holzrichter (U.S. Patent Number US 5729694 A), and further in view of PETERSEN (U.S. Patent Number US 20210373659 A1).
As to Claim 3, Liu in view of Bakish and further in view of Holzrichter teaches 3. (Original) The device according to claim 1,
Liu in view of Bakish and further in view of Holzrichter do not specifically teach wherein the bracket comprises a spectacle frame. However, PETERSEN does teach this limitation. (see PETERSEN Figure 1, Element 51, see [0029] “Preferably, the at least one surface emitter having a photodiode integrated therein is arranged on a spectacle frame and/or on a spectacle temple. In this context, a spectacle frame is especially regarded as a region of the smart glasses that surrounds the spectacle lens, and a spectacle temple is especially regarded as a holding temple that is connected to the spectacle frame, for example extending to an ear of the user. For example, several surface emitters having integrated photodiodes can distributedly be arranged around the spectacle lens on the spectacle frame, allowing particularly precise scanning of the eye over its entire range of motion. Alternatively or additionally, one or more laser sources may be integrated, preferably molded, into the spectacle lens.”
Liu in view of Bakish and further in view of Holzrichter and PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include wherein the bracket comprises a spectacle frame of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 5, Liu in view of Bakish and further in view of Holzrichter teaches 5. (Original) The device according to claim 1,
Liu in view of Bakish and further in view of Holzrichter do not specifically teach wherein the optical sensing head comprises an emitter configured to direct coherent light toward the face and an array of sensors configured to sense a secondary speckle pattern due to reflection of the coherent light from the face. However, PETERSEN does teach this limitation. (see PETERSEN [0011] “The laser feedback interferometry is considered as the detection and analysis of the overlap of the irradiated laser beam and its backscattered portion, i.e. a detection and analysis of a resulting interference-radiation. Based on laser feedback interferometry, an optical path length of the emitted laser beam is determined. The optical path length is considered to be the product of a geometric distance covered by the emitted laser beam from the laser source to the surface of the eye and a refractive index of the material present there. This means that if the laser beam is emitted in air (refractive index of approx. 1) from a laser source directly towards the eye, the optical path length is a very good approximation of the distance between the laser source and the eye. For example, if the wavelength of the emitted laser radiation is known, the optical path length can be estimated based on constructive or destructive interference.”)
Liu in view of Bakish and further in view of Holzrichter and PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include the optical sensing head comprises an emitter configured to direct coherent light toward the face and an array of sensors configured to sense a secondary speckle pattern due to reflection of the coherent light from the face of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 6, Liu in view of Bakish and further in view of Holzrichter and further in vewi of PETERSEN teaches 6. (Original) The device according to claim 5,
Furthermore, PETERSEN teaches wherein the emitter is configured to direct multiple beams of the coherent light toward different, respective locations on the face, and the array of sensors is configured to sense the secondary speckle pattern reflected from the locations. (see PETERSEN [0011] “The laser feedback interferometry is considered as the detection and analysis of the overlap of the irradiated laser beam and its backscattered portion, i.e. a detection and analysis of a resulting interference-radiation. Based on laser feedback interferometry, an optical path length of the emitted laser beam is determined. The optical path length is considered to be the product of a geometric distance covered by the emitted laser beam from the laser source to the surface of the eye and a refractive index of the material present there. This means that if the laser beam is emitted in air (refractive index of approx. 1) from a laser source directly towards the eye, the optical path length is a very good approximation of the distance between the laser source and the eye. For example, if the wavelength of the emitted laser radiation is known, the optical path length can be estimated based on constructive or destructive interference.”)
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and of Holzrichter and Petersen to include the emitter is configured to direct multiple beams of the coherent light toward different, respective locations on the face, and the array of sensors is configured to sense the secondary speckle pattern reflected from the locations of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 7, Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN teaches 7. (Original) The device according to claim 6,
Furthermore, PETERSEN teaches wherein the locations illuminated by the beams and sensed by the array of sensors extend over a field of view having an angular width of at least 60o. (see PETERSEN [0029] Preferably, the at least one surface emitter having a photodiode integrated therein is arranged on a spectacle frame and/or on a spectacle temple. In this context, a spectacle frame is especially regarded as a region of the smart glasses that surrounds the spectacle lens, and a spectacle temple is especially regarded as a holding temple that is connected to the spectacle frame, for example extending to an ear of the user. For example, several surface emitters having integrated photodiodes can distributedly be arranged around the spectacle lens on the spectacle frame, allowing particularly precise scanning of the eye over its entire range of motion. (examiner interprets at least 60o as “over (the eyes’) entire range of motion”) Alternatively or additionally, one or more laser sources may be integrated, preferably molded, into the spectacle lens.
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter and Petersen to include the locations illuminated by the beams and sensed by the array of sensors extend over a field of view having an angular width of at least 60o of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 8, Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN teaches 8. (Original) The device according to claim 6,
Furthermore, PETERSEN teaches wherein the locations illuminated by the beams and sensed by the array of sensors extend over an area of at least 1 cm2. (see PETERSEN [0029] Preferably, the at least one surface emitter having a photodiode integrated therein is arranged on a spectacle frame and/or on a spectacle temple. In this context, a spectacle frame is especially regarded as a region of the smart glasses that surrounds the spectacle lens, and a spectacle temple is especially regarded as a holding temple that is connected to the spectacle frame, for example extending to an ear of the user. For example, several surface emitters having integrated photodiodes can distributedly be arranged around the spectacle lens on the spectacle frame, allowing particularly precise scanning of the eye over its entire range of motion. (examiner interprets 1 cm2 as “over (the eyes’) entire range of motion”) Alternatively or additionally, one or more laser sources may be integrated, preferably molded, into the spectacle lens.
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter and Petersen to include the locations illuminated by the bea ms and sensed by the array of sensors extend over an area of at least 1 cm2 of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 9, Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN teaches 9. (Original) The device according to claim 6,
Furthermore Liu teaches wherein the optical sensing head comprises multiple emitters, which are configured to generate respective groups of the beams covering different, respective areas of the face, (see Liu [0011] “According to the disclosure, the speech recognition device is to be coupled to an image capturing device for receiving images of the lips of a human subject during a speech of the subject therefrom. The speech recognition device is further to be coupled to an oral cavity detecting device for receiving a contact signal and a distance signal. The contact signal is associated with contact of the oral cavity detecting device with the tongue of the subject during the speech of the subject, and the distance signal is associated with a distance of the oral cavity detecting device from the tongue during the speech of the subject. The speech recognition device includes a lip movement analyzing module, a contact analyzing module, a distance analyzing module, and a parameter collecting module.”)
Furthermore, PETERSEN teaches and wherein the processing circuitry is configured to select and actuate a subset of the emitters without actuating all the emitters. (see PETERSEN [0053] “A particularly simple and energy-saving way of operating the smart glasses 50 is shown in FIG. 5. In FIG. 5, an operating state of the smart glasses 50 is shown, wherein the input unit and/or output unit 7 is deactivated and the eye tracking arrangement 20 is in a sleep mode. In the sleep mode, only exactly one of the surface emitters 3b of the gaze detection arrangement 20 is operable, with all other surface emitters 3a, 3c, 3d, 3e being deactivated. Through the single active surface emitter 3b, only a single laser beam 1 is directed to the eye 10 in exactly one laser port 30b, as shown in FIG. 5.”)
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter and Petersen to include the the processing circuitry is configured to select and actuate a subset of the emitters without actuating all the emitters of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 10, Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN teaches 10. (Original) The device according to claim 5,
Furthermore, Liu teaches wherein the processing circuitry is configured to detect changes in the sensed secondary speckle pattern and to generate the speech output responsively to the detected changes. (See Liu [0015] “The parameter collecting module is programmed to collect the corresponding one of the lip parameters, the corresponding one of the palate parameters and the corresponding one of the tongue parameters to generate a speech parameter set corresponding to content of the speech.”)
As to Claim 11, Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN teaches 11. (Original) The device according to claim 5,
Furthermore, Holzrichter teaches wherein the processing circuitry is configured to operate the array of sensors at a first frame rate, to sense, responsively to the signal while operating at the first frame rate, a movement of the face, and to increase the frame rate responsively to the sensed movement to a second frame rate, greater than the first frame rate, for generating the speech output. (see Holzrichter (49:14-37) “(175) The feature vectors can be used for phonetic template (i.e., pattern) matching and associated acoustic speech unit identification. Each acoustic speech unit symbol is uniquely associated with a specific articulator configuration (i.e., a phonetic articulator pattern). The formed vectors, which describe these patterns, are then compared against the library data and an identification is made using the "distance" from the code book feature vectors, and using logical operations, such as "on" or "off" for the glottal motions. In the case of speech segments with multi-phonemes, similar methods of measuring vector distances can be used. One procedure is to use the square root of the sum of the squares of all relevant vector coefficient differences. (Control coefficient distances are not used). When the distance is within a value defined by the user, an identification is defined and the related probability based upon the distance measure can be attached to the identification unit as desired. The use of a logical test operation is well known. Well defined normalization and quantization techniques for feature vectors make for well defined code book comparisons because the vectors can be instrument and speaker independent. An additional advantage is that individual-speaker rates of phoneme sequence articulation can be normalized and time aligned speech frames can be produced.”) (see Holzrichter (see Holzrichter (53:39-67) “ (204) Step 5: If phoneme to phoneme transitions are not called out by step 2, generate the transition acoustic sound units using one or more of the following: Two sequential voiced sound units are joined at the glottal closed times (i.e., the glottal zeros) of voiced speech frames, while unvoiced frames (or unvoiced-voiced frames) are joined at acoustic amplitude zeros. If transition rules are present that describe the rate of interpolation between voiced phoneme units, they are used to set the transition time frame durations and to interpolate excitation and transfer function coefficients that are modified by their relationship to another articulator condition in the preceding or following time frame. Another method of interpolation is to use diphoneme or triphoneme acoustic speech patterns, pre-stored in a code book, which are normalized to the proper intensity and speech period and which are placed, automatically between any two phonemes called for from step 2. (205) Step 6: Provide the prosody for the acoustic sounds generated during each speech time frame or combination of speech time frames. For example, use prosody rules to set the rate of sound level amplitude increase, period of constancy, or rate of amplitude decrease over several speech frames. Use prosody rules to set the pitch change from the beginning of the speech sequence to the end, as defined by phrasing and punctuation rules. Such prosody information is obtained from the text-to-speech converter, in step 2, and is used to alter the frame vectors as they are taken from the code book to meet the demands of the text being synthesized into speech.”)
Liu in view of Bakish and further and Holzrichter are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include the processing circuitry is configured to operate the array of sensors at a first frame rate, to sense, responsively to the signal while operating at the first frame rate, a movement of the face, and to increase the frame rate responsively to the sensed movement to a second frame rate, greater than the first frame rate, for generating the speech output of Holzrichter. This allows speaker rates to be normalized as recognized by Holzrichter (49:14-37) (175).
As to Claim 13, Liu in view of Bakish and further in view of Holzrichter teaches 13. (Currently amended) The device according to claim 1,
Liu in view of Bakish and further in view of Holzrichter do not specifically teach wherein the optical sensing head is held by the bracket in a position that is at least 5 mm away from a skin surface of the user. However, Peterson does teach this limitation (See Peterson Figure 1 Elements 3a-d and Element 10)
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include the optical sensing head is held by the bracket in a position that is at least 5 mm away from a skin surface of the user of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 20, Liu in view of Bakish and further in view of Holzrichter teaches 20. (Currently amended) The device according to claim 1,
Liu in view of Bakish and further in view of Holzrichter do not specifically teach and comprising a user control, which is connected to the bracket and configured to sense a gesture made by the user, wherein the processing circuitry is configured to change an operational state of the device responsively to the sensed gesture. However, Petersen does teach this limitation (see Petersen [0052] “If such eye gesture is detected based on the eye velocity and optical path length 2 determined using the eye tracking arrangement 20, the image projection unit can be operated in a predefined manner. For example, the image projection unit can be activated or deactivated to enable operation of the smart glasses 50 as required, and thus in a particularly energy-saving manner. Alternatively or additionally, the eye gesture can be used to activate operation of menu options of contents displayed in an augmented or virtual reality. For example, it is particularly convenient if a particular eye gesture, preferably upwards and/or downwards rolling of the eye 10, activates scrolling of content displayed in the augmented or virtual reality.”)
Liu in view of Bakish and further in view of Holzrichter and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include and comprising a user control, which is connected to the bracket and configured to sense a gesture made by the user, wherein the processing circuitry is configured to change an operational state of the device responsively to the sensed gesture of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
Claims 14 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of Holzrichter (U.S. Patent Number US 5729694 A), and further in view of Oba (U.S. Patent Number US 7676372 B1).
As to Claim 14, Liu in view of Bakish and further in view of Holzrichter teaches 14. (Currently amended) The device according to claim 1,
Liu in view of Bakish and further in view of Holzrichter do not specifically teach and comprising one or more electrodes configured to contact a skin surface of the user, wherein the processing circuitry is configured to generate the speech output responsively to the electrical activity sensed by the one or more electrodes together with the signal output by the optical sensing head. However, Oba does teach this limitation (see Oba (24:4-19) “(146) Moreover, the HA 1 may be provided with a cochlear implant system such that recognition results can be presented to the user through a cochlear implant. That is, with this HA 1, speech information can be sent as an electrical signal to a cochlear implant system comprising, for example, an embedded electrode and a speech processor, for presentation of speech information to the user. (147) Also, the present HA 1 may be provided with an auditory brainstem implant (ABI) system comprising an electrode in contact with the cochlear nucleus (the junction of the cochlear nerves in the myelin cephalon) to send recognition results through the electrode to the user. In this manner, speech information can be sent to the user through an ABI. That is, the HA 1 can send speech information as an electrical signal through an ABI system comprising an embedded electrode, and a speech processor, etc., for presentation to the user.”)
Liu in view of Bakish and further in view of Holzrichter and Oba are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include and comprising one or more electrodes configured to contact a skin surface of the user, wherein the processing circuitry is configured to generate the speech output responsively to the electrical activity sensed by the one or more electrodes together with the signal output by the optical sensing head of Oba. This allows for the system to be easier for the user to operate as recognized by OBA (28:29-30).
As to Claim 21, Liu in view of Bakish and further in view of Holzrichter teaches 21. (Currently amended) The device according to claim 1,
Liu in view of Holzrichter do not specifically teach and comprising a speaker configured to fit in the ear of the user, wherein the processing circuitry is configured to synthesize an audio signal corresponding to the speech output for playback by the speaker. (see Oba (3:59-67) “(18) For the earphone, any magnetic earphone may be used. Any suitable commonly used microphone and earphone may be used in a sound enhancement unit or in the HA. Similarly, any suitable microphone commonly used in a middle ear implant, a cochlear implant, an ABI (auditory brainstem implant), a tactile aid or a bone-conduction ultrasound system, may be used. As the sound collection component for these microphones, echo cancellers, for example, may be used.”)
Liu in view of Bakish and further in view of Holzrichter and Oba are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Holzrichter to include and comprising a speaker configured to fit in the ear of the user, wherein the processing circuitry is configured to synthesize an audio signal corresponding to the speech output for playback by the speaker of Oba. This allows for the system to be easier for the user to operate as recognized by OBA (28:29-30).
Claims 22 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of Zamora (U.S. Patent Number US 10515636 B2).
As to independent Claim 22, Liu teaches 22. (Currently amended) A method for sensing, comprising: sensing a movement of skin of a human subject in response to words articulated by the subject (see Liu Abstract “A speech recognition system is to be used on a human subject. The speech recognition system includes an image capturing device, an oral cavity detecting device and a speech recognition device. The image capturing device captures images of lips of the subject during a speech of the subject. The oral cavity detecting device detects contact with a tongue of the subject and distance from the tongue of the subject, and accordingly generates a contact signal and a distance signal. The speech recognition device processes the images of the lips and the contact and distance signals so as to obtain content of the speech of the subject.”) and responsively to the sensed movement, generating a speech output including the articulated words.(see Liu [0068] “The output device 7 includes an audio output module 71 for outputting the synthesized pronunciation signal, and a display module 72 for displaying a word according to the word data sets received from the speech recognition device 6.”)
Lui does not specifically teach on a cheek However, Bakish does teach this limitation. (see Bakish [0049] “In some embodiments which may utilize a laser microphone or optical microphone, such optical microphone (or optical sensor) and/or its components may be implemented as (or may comprise) a Self-Mix module; for example, utilizing a self-mixing interferometry measurement technique (or feedback interferometry, or induced-modulation interferometry, or backscatter modulation interferometry), in which a laser beam is reflected from an object, back into the laser. The reflected light interferes with the light generated inside the laser, and this causes changes in the optical and/or electrical properties of the laser. Information about the target object and the laser itself may be obtained by analyzing these changes. In some embodiments, the optical microphone or laser microphone operates to remotely detect or measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear); and/or to remotely detect or measure or estimate the direct changes in skin vibrations; rather than trying to measure indirectly an effect of spoken speech on a vapor that is exhaled by the mouth of the speaker, and rather than trying to measure indirectly an effect of spoken speech on the humidity or relative humidity or gas components or liquid components that may be produced by the mouth due to spoken speech.”)
Liu and Bakish are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method combination of Liu to include the on a cheek of the user of Bakish. This allows selection of optimal signals as recognized by Bakish [0036].
Liu in view of Bakish does not specifically teach without vocalization of the words by the subject and without contacting the skin; However Zamora does teach this limitation. (see Zamora (10:31-35) “(76) Referring now to FIG. 12, a block diagram is shown illustrating an example computing device that can recognize speech using depth information. The computing device 1200 may be, for example, a laptop computer, desktop computer, tablet computer, mobile device, or wearable device,”) (see Zamora (14:21-33) “(100) Example 11 is a method for recognizing speech using depth information. The method includes receiving, via a processor, depth information corresponding to a face. The method includes detecting, via the processor, the face including lips and tracking, via the processor, a plurality of descriptor points including lip descriptor points located around the lips. The method includes calculating, via the processor, a plurality of descriptor features based on the tracked descriptor points. The method includes generating, via the processor, a visual pattern of the descriptor features over time. The method includes recognizing, via the processor, speech based on the generated visual pattern.”)(see Zamora Figure 8 elements 802, 804 and 806)
Liu in view of Bakish and Zamora are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Liu and Bakish to include the without vocalization of the words by the subject and without contacting the skin; of Zamora. This allows for improved accuracy as recognized by Zamora (2:58-60).
As to Claim 23, Liu in view of Bakish and Further in view of Zamora teaches 23. (Currently amended) The method according to claim 22,
Furthermore Bakish teaches wherein sensing the movement comprises sensing light reflected from the cheek of the subject. (see Bakish [0049] “In some embodiments which may utilize a laser microphone or optical microphone, such optical microphone (or optical sensor) and/or its components may be implemented as (or may comprise) a Self-Mix module; for example, utilizing a self-mixing interferometry measurement technique (or feedback interferometry, or induced-modulation interferometry, or backscatter modulation interferometry), in which a laser beam is reflected from an object, back into the laser. The reflected light interferes with the light generated inside the laser, and this causes changes in the optical and/or electrical properties of the laser. Information about the target object and the laser itself may be obtained by analyzing these changes. In some embodiments, the optical microphone or laser microphone operates to remotely detect or measure or estimate vibrations of the skin (or the surface) of a face-point or a face-region or a face-area of the human speaker (e.g., mouth, mouth-area, lips, lips-area, cheek, nose, chin, neck, throat, ear); and/or to remotely detect or measure or estimate the direct changes in skin vibrations; rather than trying to measure indirectly an effect of spoken speech on a vapor that is exhaled by the mouth of the speaker, and rather than trying to measure indirectly an effect of spoken speech on the humidity or relative humidity or gas components or liquid components that may be produced by the mouth due to spoken speech.”)
Liu in view of Bakish and Zamora are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method combination of Liu and Bakish and of Zamora to include the optical sensing head is configured to sense the light reflected from a cheek of the user of Bakish. This allows selection of optimal signals as recognized by Bakish [0036].
Claims 24-28 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of Zamora (U.S. Patent Number US 10515636 B2), and further in view of PETERSEN (U.S. Patent Number US 20210373659 A1).
As to Claim 24, Liu in view of Bakish and further in view of Zamora teaches 24. (Currently amended) The method according to claim 23,
Liu in view of Bakish and further in view of Zamora do not specifically teach wherein sensing the light comprises directing coherent light toward the skin and sensing a secondary speckle pattern due to reflection of the coherent light from the skin. However, PETERSEN does teach this limitation. (see PETERSEN [0011] “The laser feedback interferometry is considered as the detection and analysis of the overlap of the irradiated laser beam and its backscattered portion, i.e. a detection and analysis of a resulting interference-radiation. Based on laser feedback interferometry, an optical path length of the emitted laser beam is determined. The optical path length is considered to be the product of a geometric distance covered by the emitted laser beam from the laser source to the surface of the eye and a refractive index of the material present there. This means that if the laser beam is emitted in air (refractive index of approx. 1) from a laser source directly towards the eye, the optical path length is a very good approximation of the distance between the laser source and the eye. For example, if the wavelength of the emitted laser radiation is known, the optical path length can be estimated based on constructive or destructive interference.”)
Liu in view of Bakish and further in view of Zamora and PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora to include the optical sensing head comprises an emitter configured to direct coherent light toward the face and an array of sensors configured to sense a secondary speckle pattern due to reflection of the coherent light from the face of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 25, Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN teaches 25. (Currently amended) The method according to claim 24,
Furthermore, PETERSEN teaches wherein directing the coherent light comprises directing multiple beams of the coherent light toward different, respective locations on the face, and sensing the secondary speckle pattern reflected from each of the locations using an array of sensors. (see PETERSEN [0011] “The laser feedback interferometry is considered as the detection and analysis of the overlap of the irradiated laser beam and its backscattered portion, i.e. a detection and analysis of a resulting interference-radiation. Based on laser feedback interferometry, an optical path length of the emitted laser beam is determined. The optical path length is considered to be the product of a geometric distance covered by the emitted laser beam from the laser source to the surface of the eye and a refractive index of the material present there. This means that if the laser beam is emitted in air (refractive index of approx. 1) from a laser source directly towards the eye, the optical path length is a very good approximation of the distance between the laser source and the eye. For example, if the wavelength of the emitted laser radiation is known, the optical path length can be estimated based on constructive or destructive interference.”)
Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora and Petersen to include the emitter is configured to direct multiple beams of the coherent light toward different, respective locations on the face, and the array of sensors is configured to sense the secondary speckle pattern reflected from the locations of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 26, Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN teaches 26. (Currently amended) The method according to claim 25,
Furthermore, PETERSEN teaches wherein the locations illuminated by the beams and sensed by the array of sensors extend over a field of view having an angular width of at least 60o. (see PETERSEN [0029] Preferably, the at least one surface emitter having a photodiode integrated therein is arranged on a spectacle frame and/or on a spectacle temple. In this context, a spectacle frame is especially regarded as a region of the smart glasses that surrounds the spectacle lens, and a spectacle temple is especially regarded as a holding temple that is connected to the spectacle frame, for example extending to an ear of the user. For example, several surface emitters having integrated photodiodes can distributedly be arranged around the spectacle lens on the spectacle frame, allowing particularly precise scanning of the eye over its entire range of motion. (examiner interprets at least 60o as “over (the eyes’) entire range of motion”) Alternatively or additionally, one or more laser sources may be integrated, preferably molded, into the spectacle lens.
Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora and Petersen to include the locations illuminated by the beams and sensed by the array of sensors extend over a field of view having an angular width of at least 60o of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 27, Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN teaches 27. (Currently amended) The method according to claim 25
Furthermore, PETERSEN teaches wherein the locations illuminated by the beams and sensed by the array of sensors extend over an area of at least 1 cm2 on a cheek of the subject. (see PETERSEN [0029] Preferably, the at least one surface emitter having a photodiode integrated therein is arranged on a spectacle frame and/or on a spectacle temple. In this context, a spectacle frame is especially regarded as a region of the smart glasses that surrounds the spectacle lens, and a spectacle temple is especially regarded as a holding temple that is connected to the spectacle frame, for example extending to an ear of the user. For example, several surface emitters having integrated photodiodes can distributedly be arranged around the spectacle lens on the spectacle frame, allowing particularly precise scanning of the eye over its entire range of motion. (examiner interprets 1 cm2 as “over (the eyes’) entire range of motion”) Alternatively or additionally, one or more laser sources may be integrated, preferably molded, into the spectacle lens.
Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora and Petersen to include the locations illuminated by the beams and sensed by the array of sensors extend over an area of at least 1 cm2 of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
As to Claim 28, Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN teaches 28. (Currently amended) The method according to claim 24,
Furthermore, Peterson teaches wherein generating the speech output comprises detecting changes in the sensed secondary speckle pattern and generating the speech output responsively to the detected changes. (see PETERSEN [0011] “The laser feedback interferometry is considered as the detection and analysis of the overlap of the irradiated laser beam and its backscattered portion, i.e. a detection and analysis of a resulting interference-radiation. Based on laser feedback interferometry, an optical path length of the emitted laser beam is determined. The optical path length is considered to be the product of a geometric distance covered by the emitted laser beam from the laser source to the surface of the eye and a refractive index of the material present there. This means that if the laser beam is emitted in air (refractive index of approx. 1) from a laser source directly towards the eye, the optical path length is a very good approximation of the distance between the laser source and the eye. For example, if the wavelength of the emitted laser radiation is known, the optical path length can be estimated based on constructive or destructive interference.”)
Liu in view of Bakish and further in view of Zamora and further in view of PETERSEN are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and of Zamora and Petersen to include wherein generating the speech output comprises detecting changes in the sensed secondary speckle pattern and generating the speech output responsively to the detected changes. of PETERSEN This allows flexible use of the glasses as recognized by PETERSEN [0023].
Claims 29 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Number US 20160027441 A1) in view of Bakish (U.S. Patent Number US 20180132043 A1), and further in view of Zamora (U.S. Patent Number US 10515636 B2), and further in view of Begum (U.S. Patent Number US 20160093284 A1).
As to Claim 29, Liu in view of Bakish and further in view of Zamora teaches 29. (Currently amended) The method according to claim 22,
Liu in view of Bakish and further in view of Zamora do not specifically teach wherein generating the speech output comprises synthesizing an audio signal corresponding to the speech output. However, Begum does teach this limitation (see Begum [0017] “The words 110 are then further processed by a linguistic analysis module 116 to phonetic transcriptions to each of the words 110. The linguistic analysis module 116 divides and marks text into prosodic units, like phrases, clauses, and sentences. The process of assigning phonetic transcriptions to words is called text-to-phoneme or grapheme-to-phoneme conversion. Phonetic transcriptions and prosody information together make up the symbolic linguistic representation that is output, and referred to as phonemes 120.”) (see Begum [0018] “The phonemes 120 are input into a waveform synthesis module 130, which also receives sound frequency information 134 (e.g., sound frequencies) from an articulatory system model 140. The information 134 generated by the articulatory system model 140 and the phonemes 120 is synthesized using the waveform synthesis module 130 (e.g., wave synthesizer) to produce a speech output 150. Waveform synthesis is described in chapter 6 and shown in FIGS. 6.1-6.6 of the Levison et al. reference.”)
Liu in view of Bakish and further in view of Zamora and further in view of Begum are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora to include generating the speech output comprises synthesizing an audio signal corresponding to the speech output of Begum This allows for tailored sound frequencies as recognized by Begum [0020].
As to Claim 30, Liu in view of Bakish and further in view of Zamora teaches 30. (Currently amended) The method according to claim 22,
Liu in view of Bakish and further in view of Zamora do not specifically teach wherein generating the speech output comprises transcribing the words articulated by the subject. However, Begum does teach this limitation (see Begum [0017] The words 110 are then further processed by a linguistic analysis module 116 to phonetic transcriptions to each of the words 110. The linguistic analysis module 116 divides and marks text into prosodic units, like phrases, clauses, and sentences. The process of assigning phonetic transcriptions to words is called text-to-phoneme or grapheme-to-phoneme conversion. Phonetic transcriptions and prosody information together make up the symbolic linguistic representation that is output, and referred to as phonemes 120.” (see Begum [0018] The phonemes 120 are input into a waveform synthesis module 130, which also receives sound frequency information 134 (e.g., sound frequencies) from an articulatory system model 140. The information 134 generated by the articulatory system model 140 and the phonemes 120 is synthesized using the waveform synthesis module 130 (e.g., wave synthesizer) to produce a speech output 150. Waveform synthesis is described in chapter 6 and shown in FIGS. 6.1-6.6 of the Levison et al. reference.”)
Liu in view of Bakish and further in view of Zamora and further in view of Begum are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device combination of Liu and Bakish and of Zamora to include generating the speech output comprises transcribing the words articulated by the subject of Begum This allows for tailored sound frequencies as recognized by Begum [0020].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTEN MICHELLE MASTERS whose telephone number is (703)756-1274. The examiner can normally be reached M-F 8:30 AM - 5:00 PM.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTEN MICHELLE MASTERS whose telephone number is (703)756-1274. The examiner can normally be reached M-F 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTEN MICHELLE MASTERS/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights