Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
Examiner has cited particular columns and line numbers or figures in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to
fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Claim Rejections - 35 USC § 103
Claim(s) 161-166, 168-175, 177 and 179-180 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2017/0344812) and in view of Shillingford et al. (US 2021/0110831).
As to claim 161, Kim teaches a non-transitory computer-readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for determining subvocalized phonemes from facial skin micromovements, the operations comprising:
controlling at least one coherent light source 134 in a manner enabling illumination of a first region of a face and a second region of the face (Par.97, the plurality of light emitting parts may respectively emit infrared rays to different objects, and the plurality of light receiving parts may respectively receive infrared rays reflected by different objects);
performing pattern analysis on light reflected from the first region of the face to determine first micromovements/deformation of facial skin in the first region of the face and the second region of the face to determine second micromovements of facial skin in the second region of the face (Par.17, the light receiving part may include a plurality of light receiving parts. The plurality of light receiving parts may receive infrared rays that are sequentially emitted from the plurality of light emitting parts. For example, the plurality of light receiving parts receives infrared rays when a first light emitting part among the plurality of light emitting parts emits infrared rays, and then receives infrared rays when a second light emitting part emits infrared rays, followed by receiving infrared rays when a nth light emitting part emits infrared rays. For example, the case in which the light receiving part receives infrared rays emitted from the first light emitting part and the case in which the light receiving part receives infrared rays emitted from the second light emitting part are related to a correlation with an object, and infrared ray transmission characteristics included in the infrared rays received by the light receiving part may be changed.)
and using the first micromovements of the facial skin in the first region of the face and the second micromovements of the facial skin in the second region of the face to ascertain at least one expression or command (a transmission characteristic determination unit for determining transmission characteristic change in the received infrared rays based on the intensity information; a deformation of skin measurement unit for measuring deformation of skin for the at least one object based on the intensity information and the transmission characteristic change and a facial expression recognition unit for recognizing a facial expression by comparing the level or magnitude of the deformation of skin with pre-inclined reference data of the facial expression.(Pars.103-105, 116-120, 68-87; Figs.1-6).
PNG
media_image1.png
322
332
media_image1.png
Greyscale
It is noted that Kim doesn’t explicitly teach the facial micromovements into phonemes. However, Shillingford teaches a visual speech recognition comprising receiving a video comprising a plurality of video frames, wherein each video frame depicts a face of a person, processing the video plurality of frames and determining a phoneme expressed around the lips of the person (Pars.5-6, 25, 42-46; Figs.1-6).
PNG
media_image2.png
216
672
media_image2.png
Greyscale
The combination of the analogous arts would be obvious to one of ordinary skill in the art before the time of applicant’s invention for the purpose of converting the recognized facial expressions into words thereby improving the recognition of the user commands.
As to claim 162, Kim teaches wherein performance of the second pattern analysis occurs (sequentially) after performing the first pattern analysis (Pars.62-64, 116-117).
As to claim 163, Kim teaches wherein performance of the second pattern analysis occurs simultaneously with performance of the first pattern analysis (Pars.62-64;116-119, 134-135).
As to claim 164, Kim teaches wherein the first region is spaced apart from the second region (Fig.6; Pars.95, 116).
As to claim 165, Shillingford teaches wherein ascertaining the at least one subvocalized phoneme includes ascertaining a sequence of phonemes, and wherein the operations further include extracting meaning from the sequence of phonemes (Fig.2; Pars.6-7, 26).
As to claim 166, Shillingford teaches wherein each phoneme in the sequence of phonemes is derived from the first pattern analysis and the second pattern analysis (Fig.2).
As to claim 168, Kim teaches wherein the operations further comprise determining both the first micromovements and the second micromovements during a common time period (Pars.104-105, 122-124; Figs.4-5).
As to claim 169, Kim teaches wherein the operations further comprise receiving the first light reflections and the second light reflections via at least one detector, wherein the at least one detector and the at least one light source are integrated within a wearable housing (Fig.6).
As to claim 170, Kim teaches controlling the at least one light source includes projecting differing light patterns on the first region and the second region (Pars.116-117).
As to claim 171, Kim teaches wherein the differing light patterns include a plurality of light spots, such that the first region of the face is illuminated by at least a first light spot and the second region of the face is illuminated by at least a second light spot, different from the first light spot (Pars.116-119).
As to claim 172, Kim teaches wherein controlling the at least one light source includes illuminating the first region and the second region with a common light spot (Pars.96-98, 118-119)
As to claim 173, Kim teaches wherein the facial skin analyzed to determine the micromovement include multitude of facial muscles (Fig.6) inherently suggesting the inclusion of at least one of the cited facial muscle in the claim.
As to claim 174, Shillingford teaches wherein the operations further include accessing a default language of an individual associated with the facial skin micromovements, and using the default language to extract meaning from the at least one subvocalized phoneme (Pars.71-72; Figs.1-2)
As to claim 175, Shillingford teaches wherein the operations further include using a synthesized voice to generate an audio output reflective of the at least one subvocalized phoneme (Pars.7, 104-108).
As to claim 177, Kim teaches where wherein the operations further include determining an emotional state of an individual associated with the facial skin micromovements, Fig.8, and Shillingford teaches extracting meaning from the at least one subvocalized phoneme and the determined emotional state (Figs.1-2).
Regarding claims 179-180, the corresponding method and system comprising the steps cited in claim 161, are analogous therefore rejected as being unpatentable over Kim and in view of Shillingford for the foregoing reasons.
Allowable Subject Matter
Claims 167, 176 and 178 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Claim 167 is allowable because the prior arts do not teach wherein the operations further include identifying as private at least one phoneme in the sequence of phonemes and omitting generation of an audio output reflective of the at least one private phoneme.
Claim 176 is allowable because the prior arts do not teach wherein the at least one phoneme includes a sequence of phonemes, and wherein the operations further include determining a prosody associated with the sequence of phonemes, and extracting meaning based on the determined prosody.
Claim 178 is allowable because the prior arts do not teach wherein the operations further include identifying at least one extraneous phoneme as part of a filler and omitting generation of an audio output reflective of the extraneous phoneme.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Harada et al. (US 6,272,466) Figs.1-2.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL DEMELASH ABEBE whose telephone number is (571)272-7615. The examiner can normally be reached monday-friday 7-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at 571-272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL ABEBE/ Primary Examiner, Art Unit 2657
Prosecution Insights