Detailed Action
This communication is in response to the Arguments and Amendments filed on 1/27/2026.
Claims 1, 3-5 are pending and have been examined. Hence this action has been made Final
Claims 1, 3 and 4 are independent Device, Method and Storage Medium claims, respectively.
Apparent priority: 2/7/2023.
Any previous objection/rejection not mentioned in this Office Action has been withdrawn by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Amendment
The Applicant has amended the claims to include “receive audio data from a server; input the audio data to a machine leaning model that outputs an estimated emotion value to obtain the estimated emotion value for the input audio data;” and “determine whether the estimated emotion value is within a predetermined range around a threshold of the emotion value defined by an upper value above the threshold and a lower value below the threshold, the threshold indicating a positive emotion or a negative emotion; and” “value is within the predetermined range around the threshold, transmit a set of audio data for annotation, to an annotator, the set of audio data including the input audio data, first audio data having an emotion value of the upper value, and second audio data having an emotion value of the lower value.”
Regarding the 35 U.S. C. 101 rejection, The Applicant notes the claim, as a whole, recites information processing for outputting a set of audio data for annotation, based on input audio data.
Additionally, the claims recite inputting the audio data to a machine leaning model that outputs an estimated emotion value to obtain the estimated emotion value for the input audio data, and determining whether the estimated emotion value is within a predetermined range around a threshold of the emotion value defined by an upper value above the threshold and a lower value below the threshold, the threshold indicating a positive emotion or a negative emotion.
Examiner notes while Applicant contends that Claim 1 recites “functions that are not human activities” and is merely “information processing for outputting a set of audio data for annotation,” the recited operations—receiving audio from a server; inputting the audio to a machine-learning model to compute an estimated emotion value; determining whether that numeric output lies within a predetermined range around a positive/negative threshold; and transmitting a set including the input audio and upper/lower reference clips for annotation—are directed to abstract ideas under the USPTO’s groupings of mathematical concepts and mental processes, with the receiving/transmitting steps constituting insignificant data-gathering and post-solution output (see Alice, 573 U.S. 208; Parker v. Flook, 437 U.S. 584; Electric Power Group, 830 F.3d 1350; SAP v. InvestPic, 898 F.3d 1161; MPEP §§ 2106.04(a), 2106.05(g)). Characterizing the threshold as indicating “positive” or “negative” emotion and framing the workflow as “information processing” are field-of-use and results-oriented limitations that do not integrate the abstract idea into a practical application or improve computer functionality.
The Applicant’s arguments and amendments do not overcome the 35 U.S. C. 101 rejection.
Regarding the Claim Rejections - 35 U.S.C.§103 Applicant notes
The Office has rejected claims 1-4, under 35 U.S.C. §103, over U.S. 2020/0075040 ("Provost"), in view of U.S. 2020/0251104 ("Smith"). Applicant respectfully traverses the rejection.
Claim 1 recites that the processor is configured to determine whether the estimated emotion value is within a predetermined range around a threshold of the emotion value defined by an upper value above the threshold and a lower value below the threshold, the threshold indicating a positive emotion or a negative emotion. In a case in which the estimated emotion value is within the predetermined range around the threshold, transmit a set of audio data for annotation to a computer device of an annotator, the set of audio data including the input audio data, first audio data having an emotion value of the upper value, and second audio data having an emotion value of the lower value. The features above are not disclosed by the combination of Provost and Smith. Provost is directed to an automatic speech-based longitudinal emotion and mood recognition and describes about emotion values ([0022]), capturing audio from telephone conversations of a user ([0041]) to collect speeches ([0048]), segmenting the speeches, and asking listeners to transcribe or annotate the emotion content of the segmented speeches ([0050]). While Provost discloses that an emotion/mood state is determined based on the activation value and/or the valence value of the audio, ([0096]-[0097]). Provost does not "determine whether the estimated emotion value is within a predetermined range around a threshold of the emotion value", as recited in claim 1. Provost also does not disclose transmitting three items of audio data based on the emotion value of the input audio data, as recited in claim 1.
Examiner notes Provost determines whether emotion values fall within predefined or learned ranges corresponding to mood states. This fits the claims as amended. The claims do not overcome the previously applied prior art. Updated mappings to reflect the amendments have been made below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-5 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The independent Claims are directed to statutory categories:
Claim 1 is a Device claim and directed to the machine or manufacture category of patentable subject matter.
Claim 3 is a Method claim and is directed to the machine or manufacture category of patentable subject matter.
Claim 4 is a Storage Medium claim and is directed to the machine or manufacture category of patentable subject matter.
Independent claim 1 recites,
“1. An annotation requesting device comprising: a memory; and a processor coupled to the memory, the processor being configured to:
receive audio data from a server; [this relates to a human receiving audio through the auditory system]
input the audio data to a machine leaning model that outputs an estimated emotion value to obtain the estimated emotion value for the input audio data; [this relates to a human estimating emotion using empathy]
determine whether the estimated emotion value is within a predetermined range around a threshold of the emotion value defined by an upper value above the threshold and a lower value below the threshold, the threshold indicating a positive emotion or a negative emotion; and (This relates to a human using the human auditory system to receive audio and a human using natural language understanding and empathy detection in the human mind to identify emotions of a speaker, including positive and negative emotions.)
in a case in which the estimated emotion value is positioned within the predetermined range around the threshold, transmit a set of audio data for annotation to an annotator, (This relates to a human using speech or pen and paper to request an annotation from an annotator.) the set of audio data including the input audio data first audio data having an emotion value of the upper value, and second audio data having an emotion value of the lower value., (This relates to a human estimating an emotion is within a threshold.)
The Dependent Claims do not include additional limitations that could incorporate the abstract idea into a practical application or cause the Claim as a whole to amount to significantly more than the underlying abstract idea.
Regarding Independent Claim 3, Claim 3 is a Method claim with limitations similar to Claim 1, and is rejected under the same rationale.
Regarding Independent Claim 4, Claim 4 is a Storage Medium claim with limitations similar to Claim 1, and is rejected under the same rationale.
This judicial exception is not integrated into a practical application. In particular, claims 1, 3 and 4 recite additional elements of “processor”, “memory”, and “computer”. For example, in [0021] of the as filed specification, there is description of using a general computer including a central processing unit (CPU) 11A, a read only memory (ROM) 11B, a random access memory (RAM) 11C, a storage 11D, an interface (I/F) 11E, a bus 11F, and Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using a computer is noted as a general computer. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Further, the additional limitation in the claims noted above are directed towards insignificant solution activity. The claims are not patent eligible.
Dependent claim 5 recites,
“5. (New) The annotation requesting device according to claim 1, wherein the first and second audio data comprising audio data to which the emotion value has been previously estimated by the machine leaning model, and/or audio data that has been annotated. (This relates to a human using speech to present audio above and below a threshold).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Provost (U.S. Patent Number US 20200075040 A1) in view of Smith (U.S. Patent Number US 20200251104 A1).
Regarding Claim 1, Provost teaches 1. An annotation requesting device comprising: a memory; and a processor coupled to the memory, the processor being configured to: (see Provost [0007] “In another aspect, a mood state prediction system may include a first computing device comprising a processor, a microphone, and a non-transitory memory,…”) receive audio data from a server; (see provost [0046] “…server…”) (see Provost [0041] “…captures audio from telephone conversations of a user…”) input the audio data to a machine leaning model that outputs an estimated emotion value to obtain the estimated emotion value for the input audio data; (see Provost, [0022] “The present techniques demonstrate that emotion can simplify mood prediction by acting as an intermediary between speech (rapidly varying) and mood (slowly varying). One of the hallmark symptoms of BD is emotion dysregulation, suggesting that the tracking of emotion changes will provide important insights into an individual's mood variation. Herein, emotion values may be defined in terms of valence (positive vs. negative) and activation (calm vs. excited), both of which are observable from expressed behaviors such as speech.”) (see Provost, [0041] “… In an embodiment, a method and system may include (1) a deployable smartphone app that captures audio from telephone conversations of a user (e.g., the user 108) and securely stores and transmits the audio data and (2) computational strategies to detect mood. The present techniques may include technology to measure emotion expressed and experienced using ambient microphone sensing and to predict mood from emotions and language measured from cellphone conversations and ambient audio. Expressed emotions may be measured in the ambient social environment. Ambient audio may be sampled to detect: (1) who is speaking, the participant or someone else (speaker verification) and (2) the valence and activation (emotion recognition). Together, these components may measure the emotions that an individual expresses and experiences. Speech activity detection (e.g., speech/non-speech) may be performed, as in bubble 206, and only speech regions are processed (e.g., when ratio of speech as computed at bubble 208 is above a set threshold).”) (see Provost [0044] “Emotion recognition 216 may also be implemented using other ML techniques (e.g., deep learning), “) determine whether the estimated emotion value is within a predetermined range around a threshold of the emotion value defined by an upper value above the threshold and a lower value below the threshold, the threshold indicating a positive emotion or a negative emotion; (see Provost, [0097] “Method 600 may include determining an emotion state of the user based on the activation value and/or the valence value (block 608). The emotion state may include a diagnostic boolean (bipolar/non-bipolar) or scalar value representing a correlation coefficient corresponding to manic and/or depressed states. Pre-determined thresholds may be applied to the output of the ML models. …”) and in a case in which the estimated emotion value is within the predetermined range around the threshold, transmit a set of audio data for annotation, to an annotator, (see Provost, [0050] “An emotion and language annotation and transcription pipeline (e.g., the pipeline 200 of FIG. 2A) may be used to support the development of emotion classifiers and speech recognition systems for the cellphone calls. Segments of contiguous speech may be extracted from both the personal and assessment calls, ranging in length from 3-30 seconds. A subset of the segments may be selected randomly, following a Gaussian distribution centered at the assessment call, wherein priority is placed on segments most closely associated with clinical assessment information. Listeners may be asked to either transcribe the speech (language) or to annotate the emotion content of the speech, rating their perception of valence and activation using a standardized rating system. Many segments (e.g., tens of thousands or more per year from study participants) may be transcribed and annotated, with at least one transcription and five emotion annotations per segment.”)
Provost does not specifically teach the set of audio data including the input audio data, first audio data having an emotion value of the upper value, and second audio data having an emotion value of the lower value. However, Smith teaches this limitation. (See Smith [0041] “The speech quality detector 285 may be capable of determining various qualities of the speech represented in the audio data 211. Such qualities include, for example, whether the speech was whispered, whether the speech was spoken in an excited voice, whether the speech was spoken in a sad voice, whether the speech was whined, whether the speech was shouted, etc. The speech quality detector 285 may output a speech quality indicator 285 representing the one or more speech qualities of the speech in the audio data 211. Each speech quality in the speech quality indicator may be associated with a confidence value. Each confidence value may be a binned value (e.g., high, medium, low) or numeric value (e.g., a value from 0 to 1, or some other scale). (examiner interprets predetermined range as “0 to 1”)Each speech quality in the speech quality indicator may alternatively be represented as a binary value (e.g., yes or no) indicating whether the speech exhibited that particular speech quality. The speech quality indicator may include values if the speech quality detector 285 assumes the audio data is comprised of high-quality signals. If such an assumption is not made, signal quality may be one factor in determine a confidence value for a particular speech quality.”) (see Smith [0107] “Nonetheless, there may be instances where the speaking user wants the content output to the recipient user to have a quality corresponding to the quality of the user's speech. The system may be configured to not ignore the speech quality indicator, even when content is to be output by a second user device, if the user explicitly indicates such. For example, if the user says “Alexa, whisper to John that I am going to be late to our meeting,” the user's indication of “whisper to” in conjunction with the speech being whispered may cause the system to output whispered content to John.”) (see Smith [0108] “A single session ID may be associated with a dialog between a single user and the system. A dialog may corresponding to various instances of user input and corresponding system output. One instance of user input and corresponding system output may correspond to one speech quality (e.g., whisper) while another instance of user input and corresponding system output may correspond to another speech quality (e.g., shout). Thus, a single session ID may include instances of different speech qualities and a speech quality of a particular output may be directly tied to only the speech quality of the corresponding input. Thus, if a user whispers a first input, the system outputs responsive whispered output, and the user thereafter shouts an input, the system may output responsive shouted output rather than responsive whispered output.”) (see Smith [0112] “The TTSFE 616 transforms text data 610 into a symbolic linguistic representation for processing by the speech synthesis engine 618. The TTSFE 616 may also process other data 615 that indicate how specific words should be pronounced, for example by indicating the desired output speech quality in tags formatted according to speech synthesis markup language (SSML) or in some other form. For example, a first tag may be included with text marking the beginning of when text should be whispered (e.g., <begin whisper>) and a second tag may be included with text marking the end of when text should be whispered (e.g., <end whisper>). The tags may be included in the text data 610 and/or the text for a TTS request may be accompanied by separate metadata indicating what text should be whispered (or have some other indicated audio characteristic). The speech synthesis engine 618 compares the annotated phonetic units, and optionally other information, stored in the TTS unit storage 672 and/or TTS vocoder storage 680 for converting the text data into audio data 690 corresponding to synthesized speech. The TTSFE 616 and the speech synthesis engine 618 may include their own controller(s)/processor(s) and memory or they may use the controller(s)/processor(s) and memory of the server(s) 120, the device 110, or other device, for example. Similarly, the instructions for operating the TTSFE 616 and the speech synthesis engine 618 may be located within the TTS component 695, within the memory and/or storage of the server(s) 120, the device 110, or within an external device.”)
Examiner notes Provost teaches some of limitation See Provost [0035] “…wherein each segment was annotated on average 4 times by a team of 11 human listeners…”
Provost and Smith are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the device of Provost to incorporate the teachings of Smith to include other audio data that has been estimated or annotated and that is positioned within the predetermined range. This allows for user intent to be more clearly defined and determined and as recognized by Smith in [0108-0109].
Regarding independent Claim 3, claim 3 is a Method claim with limitations similar to that of Claim 1 and is rejected under the same rationale. Additionally, Provost teaches 3. An annotation requesting method comprising performing processing by a computer, the processing comprising: (see Provost [0084] “…an input device including one or more peripheral device such as a detached keyboard or mouse, or an integral device such as a capacitive touch screen of a portable computing device, and/or a microphone; and/or a display device (e.g., a computer monitor, speaker, etc.).”)
Regarding independent Claim 4, claim 4 is a storage medium claim with limitations similar to that of Claim 1 and is rejected under the same rationale. Additionally, Provost teaches 4. A non-transitory storage medium storing a program that is executable by a computer to perform annotation requesting processing, the annotation requesting processing comprising: (see Provost [0007] “In another aspect, a mood state prediction system may include a first computing device comprising a processor, a microphone, and a non-transitory memory,…”)
As to Dependent Claim 5, Provost in view of Smith teaches 5. (New) The annotation requesting device according to claim 1,
Furthermore, Provost teaches wherein the first and second audio data comprising audio data to which the emotion value has been previously estimated by the machine leaning model, and/or audio data that has been annotated. (see [0035] The performance of the methods and systems may be assessed using the Area Under the Receiver Operating Characteristic Curve (AUC) metric. For example, AUCs of 0.72+/0.20 for manic states and 0.75+/−0.14 for depressed states may be determined. The approach may then be personalized by merging the subject-independent system with a system adapted to a single person. Performance may be improved significantly (e.g., 0.78+/−0.14 AUC for depression). An emotion recognition proof of concept study was performed using the collected data set. For example, 13,611 segments of speech (6-8 seconds each) from 12 participants were annotated, wherein each segment was annotated on average 4 times by a team of 11 human listeners using a discrete 9-point Likert scale (1=low; 9=high). Emotion recognition algorithms using these data were then developed, demonstrating that the algorithms' predictions were well correlated with activation and valence (e.g., Pearson's Correlation Coefficients of 0.712 and 0.405, respectively). The present techniques have also been used to demonstrate that automatic estimates of emotion correlate to mood state and mood symptom severity.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTEN MICHELLE MASTERS whose telephone number is (703)756-1274. The examiner can normally be reached M-F 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTEN MICHELLE MASTERS/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights