DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were submitted on 12/07/2023. These drawings are reviewed and accepted by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) mathematical calculations for determining change points within a conversation. This judicial exception is not integrated into a practical application because the additional elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea.
Independent claims 1, 10, and 19 recite, in part, obtaining a signal, extracting a one-dimensional feature function, applying Gaussian smoothing, identifying zero-crossing points, and determining a set of change points within a conversation. These steps merely mathematical calculations on a generic signal, which corresponds to concepts identified as abstract ideas by the courts. The computer is recited at a high level of generality and are recited as performing generic computer functions routinely used in computer applications. Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Therefore, the claims as a whole do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Dependent claims 2-9, 11-18, and 20 are rejected under the same rationale. For example, claim 2 recites performing cepstral analysis on an audio signal; claim 3 recites applying Gaussian smoothing; claim 4 recites mapping zero-crossing points; claim performs clustering; claim 6 consolidates the data points; claim 7 performs facial emotion recognition analysis on video signals; claim 8 multiplies values; and claim 9 annotates the signal. All of the method steps above are mathematical calculations (abstract ideas) that do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 5, 6, 9, 10, 14, 15, 18, 19, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Zass et al. (US 20230052442 A1).
Regarding claims 1, 10, and 20 Zass teaches:
“A computer-implemented method for detecting change points within a conversation” (par. 0055; ‘In some examples, events 124 may comprise an interaction among two or more objects, such as a conversation between two people, an interaction of a person with a computing device, a person operating a machine, rubbing of two objects, collision of two objects, and so forth.’), the method comprising:
“obtaining, by a computer, a signal associated with the conversation” (par. 0055; ‘In some examples, events 124 may comprise an interaction among two or more objects, such as a conversation between two people, an interaction of a person with a computing device, a person operating a machine, rubbing of two objects, collision of two objects, and so forth.’);
“extracting a one-dimensional (1D) feature function from the signal” (par. 0086; ‘In some examples, audio data may comprise one or more channels, and each channel may include a stream or a one-dimensional array of values. In one example, calculating a convolution of audio data may include calculating a one dimensional convolution on one or more channels of the audio data.’);
“applying Gaussian smoothing on the 1D feature function” (par. 0082; ‘For example, the transformation function may comprise convolutions, audio filters (such as low-pass filters, high-pass filters, band-pass filters, all-pass filters, etc.), linear functions, nonlinear functions, and so forth. In some examples, the audio data may be preprocessed by smoothing the audio data, for example using Gaussian convolution, using a median filter, and so forth.’);
“identifying zero-crossing points on the smoothed 1D feature function” (par. 0082; ‘Some non-limiting examples of such audio features may include: auto-correlation; number of zero crossings of the audio signal; number of zero crossings of the audio signal centroid;…’); and
“determining a set of change points within the conversation based on the identified zero-crossing points” (par. 0083; ‘In some embodiments, analyzing audio data (for example, by the methods, steps and modules described herein) may comprise analyzing the audio data and/or the preprocessed audio data using one or more rules, functions, procedures, artificial neural networks, speech recognition algorithms, speaker recognition algorithms, speaker diarisation algorithms, audio segmentation algorithms, noise cancelling algorithms, source separation algorithms, inference models, and so forth.’ Speaker diarization is the process of segmenting an audio recording by speaker, answering the question, "Who spoke when?" by assigning unique labels (like Speaker 1, Speaker 2) to different segments of speech in a conversation or meeting. Therefore, speaker diarization can be used for determining a set of change points within the conversation, i.e., change of speaker.).
Regarding claims 5 (dep. on claim 1), 14 (dep. on claim 10), and 20 (dep. on claim 19), Zass further teaches:
“applying a clustering technique to consolidate the identified zero-crossing points into a smaller set” (par. 0079; ‘In yet another example, a trained machine learning algorithm may include a clustering model, the input may include a sample, and the inferred output may include an assignment of the sample to at least one cluster.’).
Regarding claims 6 (dep. on claim 5) and 15 (dep. on claim 14), Zass further teaches:
“outputting the consolidated smaller set of zero-crossing points as the change points” (Zass: par. 0094; ‘In another example, Step 442 may receive a real-time audio stream of a conversation, and Step 444 may analyze the audio stream using a speaker diarisation algorithm to identify participants in the conversation, for example Bob and Alice. Further, Step 406 may analyze data associated with each participants (for example the portion of the audio stream including speech produced by the participant using an audio classification algorithm) to select an adjective, for example ‘suggestive’ for Bob and ‘categorical’ for Alice.’).
Regarding claims 9 (dep. on claim 1) and 18 (dep. on claim 10), Zass further teaches:
“annotating the signal using the determined set of change points” (par. 0094; ‘Further, Step 408 may generate a textual content that includes the generated descriptions of the participants, such as ‘Alice provided a categorical denial to the accusations Bob made in his suggestive voice.’ Step 410 may publish this textual content, for example in an article.’).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 3, 4, 11, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zass in view of Sumida et al. (US 20210304755 A1).
Regarding claims 2 (dep. on claim 1) and 11 (dep. on claim 10), Zass further teaches:
“wherein the signal comprises an audio signal” (par. 0082; audio data); and
“wherein extracting the 1D feature function comprises performing cepstral analysis on the audio signal to obtain one or more Mel-Frequency [[Cepstral Coefficients (MFCCs)]]” (par. 0082; ‘a log spectrogram of at least part of the audio data; a Mel-Frequency Spectrum of at least part of the audio data;…’).
However, Zass does not expressly teach MFCC.
Sumida teaches:
“wherein extracting the 1D feature function comprises performing cepstral analysis on the audio signal to obtain one or more Mel-Frequency Cepstral Coefficients (MFCCs)” (par. 0039; ‘The voice feature quantity is represented by a characteristic parameter indicating the acoustic feature of the voice in the frame. The calculated voice feature quantity is, for example, a power, a number of zero-crossings, a mel-frequency cepstrum coefficient (MFCC), and the like.’).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zass’s audio pre-processing methods by incorporating Sumida’s voice feature quantity calculation method in order to calculate a voice feature quantity for voice analysis purposes. The combination allows for determining utterance state. (Sumida: par. 0039)
Regarding claims 3 (dep. on claim 2) and 12 (dep. on claim 11), the combination of Zass in view of Sumida further teaches:
“applying the Gaussian smoothing on a Mel-Frequency Cepstral Coefficient (MFCC)” (Zass: par. 0082; ‘smoothing the audio data, for example using Gaussian convolution…’);
“determining whether a number of identified zero-crossing points on the MFCC is within a predetermined range” (Sumida: par. 0040; ‘The voice analysis part 112, for example, determines as an utterance section a frame in which the power is greater than a predetermined lower limit of the power and the number of zero-crossings is within a predetermined range (e.g., 300 to 1000 times per second), and determines the other frames as non-voice sections.’) and
“in response to the number of identified zero-crossing points on the MFCC being outside of the predetermined range, discarding the MFCC and selecting a different MFCC for processing” (Sumida: par. 0041; ‘The voice recognition part 114 performs a voice recognition processing on the voice feature quantity inputted from the voice analysis part 112 for each utterance section by using a voice recognition model stored in advance in the storage part 130.’ Voice recognition process uses utterance sections and not the non-voice sections.).
Regarding claims 4 (dep. on claim 2) and 13 (dep. on claim 11), the combination of Zass in view of Sumida further teaches:
“mapping the identified zero-crossing points on the MFCC to time instances” (Sumida: par. 0046; ‘In addition, when the utterance identification information is added to the displayed text information, the minutes creating part 122 may store to the storage part 130 the utterance identification information, in place of the date and time information or the acquisition source identification information or together with the date and time information or the acquisition source identification information, in association with the display text information.’).
Claim(s) 7, 8, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zass in view of Brownlee et al. (US 20220392485 A1).
Regarding claims 7 (dep. on claim 1) and 16 (dep. on claim 10), Zass further teaches:
“wherein the signal comprises a video signal” (par. 0067; ‘In some embodiments, the one or more image sensors 260 may be configured to capture visual information by converting light to: images; sequence of images; videos; 3D images; sequence of 3D images; 3D videos; and so forth.’); and
“wherein extracting the 1D feature function comprises performing facial [[emotion]] recognition (FER) analysis on each frame of the video signal to generate a 1D conversational vibe function associated with the video signal” (par. 0067; ‘In another example, the captured visual information may be processed in order to: detect objects, detect events, detect action, detect face, detect people, recognize person, and so forth.’).
However, Zass does not expressly teach facial remotion recognition, as “wherein extracting the 1D feature function comprises performing facial emotion recognition (FER) analysis on each frame of the video signal to generate a 1D conversational vibe function associated with the video signal.”
Brownlee teaches:
“wherein extracting the 1D feature function comprises performing facial emotion recognition (FER) analysis on each frame of the video signal to generate a 1D conversational vibe function associated with the video signal” (par. 0025; ‘As shown in FIG. 2, audio input or biometric signal input 200 (e.g., facial expression, body language etc.) or a combination thereof is fed into a signal processor where it is pre-processed prior to being fed into the machine learning model (step 202). A machine learning model is invoked in step 204 to identify the emotional parameters in the input signal.’).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zass’s video processing by incorporating Brownlee’s machine learning model in order to identify emotional parameters in the signal. The combination provides real-time emotional feedback in an audio-visual communication environment. (Brownlee: par. 0005)
Regarding claims 8 (dep. on claim 7) and 17 (dep. on claim 16), the combination of Zass in view of Brownlee further teaches:
“wherein generating the 1D conversational vibe function further comprises multiplying probability of a detected emotion with a valence value corresponding to the detected emotion” (Brownlee: par. 0022; ‘A machine learning model analyzes the pitch changes in the speech as defined by the pre-processing stage, and determines the emotion being expressed, enabling people to identify the emotional valence of those to whom they speak based on the emotional content in the speech, which, in the case of a video call may be supplemented with emotional content as extracted from the facial features and/or body language of the speaker.’).
Conclusion
Other pertinent prior art are cited in the PTO-892 for the applicant's consideration.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK VILLENA whose telephone number is (571)270-3191. The examiner can normally be reached 10 am - 6pm EST Monday through Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at (571) 272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MARK . VILLENA
Examiner
Art Unit 2658
/MARK VILLENA/Examiner, Art Unit 2658