Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 07/15/2024. Claims 1-19 are pending and have been examined. Claims 1-19 are rejected.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/15/2024 and 05/22/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The present application claims foreign priority based on KR10-2023-0091935 filed 07/14/2023. The examiner notes that a certified copy (in Korean) of the above-noted application was retrieved on 02/25/2026. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Drawings
The drawings are objected to because:
FIG. 2, S200: "SPECIFIC DATA" should be "SPECIFIC EMOTION"
FIG. 3: "tratin model" and "lebel"
FIG. 3: boxes 100, 310, and 320 are labeled as data, but the spec calls them modules
FIG. 7: two boxes are both labeled "Reddit & AI Hub data"
FIG. 8: "annoatate"
FIG. 12: "hyperssomnia," "foces," "suicidatl," "sucidal"
FIG. 15: Layer 11 panel is not described in the spec
FIG. 16A: "rmes" and the undefined "re-score"
FIG. 17: turn 6 is truncated and turn 7 is missing
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “FIG. 3 numerical 10 ” has been used to designate both FIG.1 emotion diagnosis device and FIG. 3 the language model training device. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: “1”, “language model training device 1”. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: FIG. 3 “20”. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to under 37 CFR 1.83(a) because they fail to show as described in the specification. FIG. 15: page 21 rely on line color (to show the attention head) and line thickness (to show attention score). Neither is visible in the black-and-white figure, which only has uniform dashed lines. Any structural detail that is essential for a proper understanding of the disclosed invention should be shown in the drawing. MPEP § 608.02(d). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
Page 2: the objective about "rendering points," "skeleton data," and "transmitting haptics" is unrelated to this invention and looks like it was from another application.
Page 2: "Major Depressive Episode (MDD)". It should be "Major Depressive Disorder (MDD)."
Pages 3 and 4: "the the first training dataset" should be "the first training dataset."
Page 5: the second emotion diagnosis device paragraph (mid-page) repeats word for word the one beginning at the bottom of page 6. One copy should be deleted.
Pages 5 and 6: "training language model pe," has a stray "pe"
Page 6: "dataset.As described above" is missing a space and should start a new paragraph.
Page 11: "loaded into in the memory 12" should be "loaded into the memory 12." Also, "the above-described application" has no antecedent.
Page 14: "bidirectional encoder representations transformers" should be "bidirectional encoder representations from transformers."
Page 14: "The second language model 410" should be "420."
Page 16: this says the first training dataset is "constructed in step S240," but S240 is the annotation step. Suggested fix: "In step S240, the emotion data is annotated based on classification conditions preset for the constructed first training dataset."
Page 17: "the BWS dataset constructed through FIG. 7" should likely say FIG. 6. Dataset construction is described with FIG. 6 at the top of the same page, and FIG. 7 is the annotation environment.
Page 18: "Depressed among detailed symptoms, and some data..." is a sentence fragment, so it's unclear what is extracted.
Page 18: the formula "(highest - lowest)/8 × 100" gives -100 to 100. That conflicts with the -1 to 1 intensity range stated on pages 17-18.
Page 19: "include" should be "includes."
Page 20: "symptoms #1 and #2 are necessarily included" The sentence cites FIG. 12 while describing FIG. 13's criteria.
Page 20: "Bases words and related words" should be "Base words and similar words," the term used everywhere else.
The spec reads like a literal translation throughout, with missing articles like "training language model." It needs a full proofread; the grammar items above aren't exhaustive.
Appropriate correction is required.
Claim Objections
Claims 4, 11, and 15 objected to because of the following informalities: "a collection of overall rankings of respondents" should read "a collection of overall rankings by respondents". Appropriate correction is required.
Claims 5 and 16 objected to because of the following informalities: "as emotion data from the extracted data" should read "as the emotion data from the extracted data", or the emotion data of claims 5 and 16 should otherwise be distinguished from the emotion data of claims 1 and 12 (see the rejection of claims 7 and 18 under 35 U.S.C. 112(b) above). Appropriate correction is required.
Claims 7 and 18 objected to because of the following informalities: "preset for the the first training dataset" should read "preset for the first training dataset". Appropriate correction is required.
Claims 8 and 19 objected to because of the following informalities: "obtaining the similar data and daily data as emotion data" should be amended to distinguish the emotion data of the second training dataset from the emotion data recited in claims 1 and 5 (claims 12 and 16), for example, "obtaining the similar data and daily data as second emotion data". Appropriate correction is required.
Claim 12 objected to because of the following informalities: "perform a method for training language model" should read "perform a method for training a language model". Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-11 and 17-19 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6 and 7 each recite "The method of claim 9". Claim 9 is directed to "A device for emotion diagnosis," is set forth after claims 6 and 7, and does not recite a method. Accordingly, there is insufficient antecedent basis in claims 6 and 7 for the limitations "the method" (EN: For purposes of examination, claims 6 and 7 are interpreted as depending from claim 5. See also the rejection of claims 6 and 7 under 35 U.S.C. 112(d) below.)
Claims 6 and 17 recite the limitation "the data" in the phrase "obtaining the data as daily data". There is insufficient antecedent basis for this limitation in the claims. (Examiner’s Note: Claims 6 and 17, as interpreted, incorporate "text data" and "clean data" from claims 1 and 12 and "the extracted data" from claims 5 and 16, and themselves recite "data including keywords and negative words," so it is unclear which data is obtained as the daily data.)
Claims 7 and 18 recite the limitation "the emotion data" in the phrase "matching predetermined annotation information to the emotion data". There is insufficient antecedent basis for this limitation in the claims. (Examiner’s Note: Claims 7 and 18, as interpreted, incorporate two recitations of emotion data: the "emotion data related to a specific emotion" detected from the clean data in claims 1 and 12, and the "data related to symptoms related to the specific emotion" obtained "as emotion data" from the extracted data in claims 5 and 16. Therefore it is unclear to which emotion data claim 7 and 18 is referring to.
The term "similar" in claims 8 and 19 is a relative term which renders the claim indefinite. The term "similar" is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. (Examiner’s Note: Claims 8 and 19 recite "extracting similar data including similar words from the clean data based on preset base words related to the specific emotion." The claims do not recite any measure, threshold, or criterion for how similar a word must be to a preset base word to be a "similar word," and "similar data" is defined only as data including the similar words.)
Claim 9 recites the limitation "the language model" in the phrase "output an intensity of emotion and classified detailed emotions with respect to a specific emotion from the text data using the language model". There is insufficient antecedent basis for this limitation in the claim. (Examiner’s Note: Claim 9 recites "a pre-trained language model".)
Claims 10-11 are also rejected for being dependent on claim 9.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 6 and 7 rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claims 6 and 7 each recite "The method of claim 9." Claim 9 is not a claim previously set forth and claims 6 and 7 do not specify a further limitation of the subject matter of claim 9, because claim 9 is directed to a device for emotion diagnosis comprising a memory and a processor, while claims 6 and 7 recite steps of a method for training a language model. (Examiner’s Note: It appears that claims 6 and 7 were intended to depend from claim 5, consistent with parallel claims 17 and 18, which depend from claim 16.) Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claim 1:
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A prong 1: The claim recites the following abstract ideas:
generating clean data by removing errors included in the received text data; (A person mentally or with a pen and paper can read text data and strike out errors to produce clean data.)
detecting emotion data related to a specific emotion from the clean data and generating a training dataset including the detected emotion data; and (A person mentally or with a pen and paper can read the clean data, identify text related to a specific emotion and gather the identified text into a dataset.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
A method for training a language model performed by a device for training the language model, the method comprising: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic device used as a tool to perform the abstract ideas.)
receiving text data; (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)). -- EN: Merely receiving text data, i.e., mere data gathering.)
training the language model using the training dataset. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic training of a generic language model with no additional details.)
Step 2B:
A method for training a language model performed by a device for training the language model, the method comprising: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic device used as a tool to perform the abstract ideas.)
receiving text data; (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
training the language model using the training dataset. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic training of a generic language model with no additional details.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 2:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 1 above, which claim 2 depends on. Claim 2 further recites:
...calculating an intensity of emotion with respect to the specific emotion; and (A person mentally or with a pen and paper can assign a numerical intensity to an emotion expressed in text.)
...performing detailed emotion classification with respect to the specific emotion. (A person mentally or with a pen and paper can classify the detailed emotions expressed in text.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
wherein the language model includes: a first language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model used as a tool to perform the abstract idea.)
a second language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model used as a tool to perform the abstract idea.)
Step 2B:
wherein the language model includes: a first language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model used as a tool to perform the abstract idea.)
a second language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model used as a tool to perform the abstract idea.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 3:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 1 above, which claim 3 depends on. Claim 3 further recites:
wherein the generating clean data includes: removing missing values from the text data; and (A person mentally or with a pen and paper can strike out entries having missing values from text data.)
removing errors from the text data by performing a referential integrity check on data from which the missing values have been removed. (A person mentally or with a pen and paper can check whether each data entry refers to existing data and strike out the entries that do not.)
Step 2A prong 2: The claim does not recite additional elements therefore the judicial exception is not integrated into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 4:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 1 above, which claim 4 depends on. Claim 4 further recites:
wherein the training dataset includes: a first training dataset including a collection of overall rankings of respondents; and (A person mentally or with a pen and paper can gather a collection of overall rankings of respondents.)
a second training dataset including emotion diagnosis information. (A person mentally or with a pen and paper can gather a collection of emotion diagnosis information.)
Step 2A prong 2: The claim does not recite additional elements therefore the judicial exception is not integrated into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 5:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 4 above, which claim 5 depends on. Claim 5 further recites:
wherein the generating the first training dataset includes: extracting data including keywords related to the specific emotion from the clean data; and (A person mentally or with a pen and paper can pick out text containing keywords related to the specific emotion.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
obtaining data related to symptoms related to the specific emotion as emotion data from the extracted data. (Insignificant extra-solution activity as the limitation amounts to obtaining data (MPEP 2106.05(g)). -- EN: Merely obtaining symptom-related data from the extracted data, i.e., mere data gathering.)
Step 2B:
obtaining data related to symptoms related to the specific emotion as emotion data from the extracted data. (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 6:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 5 above, which claim 6 depends on. Claim 6 further recites:
EN: Claim 6 improperly depends from device claim 9; interpreted as depending from claim 5.
wherein the generating the first training dataset includes extracting data including keywords and negative words related to the specific emotion from the clean data and... (A person mentally or with a pen and paper can pick out text containing keywords and negative words related to the specific emotion.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
...obtaining the data as daily data. (Insignificant extra-solution activity as the limitation amounts to obtaining data (MPEP 2106.05(g)). -- EN: Merely obtaining the extracted data as daily data, i.e., mere data gathering.)
Step 2B:
...obtaining the data as daily data. (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 7:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 5 above, which claim 7 depends on. Claim 7 further recites:
EN: Claim 7 improperly depends from device claim 9; interpreted as depending from claim 5.
further comprising, after the generating the first training dataset, matching predetermined annotation information to the emotion data based on classification conditions preset for the the first training dataset. (A person mentally or with a pen and paper can annotate each item of emotion data according to preset classification conditions.)
Step 2A prong 2: The claim does not recite additional elements therefore the judicial exception is not integrated into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 8:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 5 above, which claim 8 depends on. Claim 8 further recites:
wherein the generating the second training dataset includes: extracting similar data including similar words from the clean data based on preset base words related to the specific emotion; and (A person mentally or with a pen and paper can pick out text containing words similar to preset base words.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
obtaining the similar data and daily data as emotion data. (Insignificant extra-solution activity as the limitation amounts to obtaining data (MPEP 2106.05(g)). -- EN: Merely obtaining the similar data and daily data, i.e., mere data gathering.)
Step 2B:
obtaining the similar data and daily data as emotion data. (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 9:
Step 1: The claim recites a device; therefore, it is directed to the statutory category of machine.
Step 2A prong 1: The claim recites the following abstract ideas:
output an intensity of emotion and classified detailed emotions with respect to a specific emotion from the text data... (A person mentally or with a pen and paper can read text, assign an intensity to an emotion, classify the detailed emotions expressed and write down the result.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
A device for emotion diagnosis, the device comprising: a memory configured to store a pre-trained language model and one or more instructions for executing the pre-trained language model; and a processor configured to execute the one or more instructions stored in the memory, wherein the instructions, when executed by the processor, cause the processor to: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic, off the shelf processor and memory used as tools to perform the abstract idea.)
receive text data for diagnosing an emotion, and (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)). -- EN: Merely receiving text data, i.e., mere data gathering.)
...using the language model, (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model used as a tool to perform the abstract idea.)
wherein the pre-trained language model has been trained using a training dataset on emotion data related to the specific emotion. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model trained in a generic manner with no additional details.)
Step 2B:
A device for emotion diagnosis, the device comprising: a memory configured to store a pre-trained language model and one or more instructions for executing the pre-trained language model; and a processor configured to execute the one or more instructions stored in the memory, wherein the instructions, when executed by the processor, cause the processor to: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic, off the shelf processor and memory used as tools to perform the abstract idea.)
receive text data for diagnosing an emotion, and (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
...using the language model, (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model used as a tool to perform the abstract idea.)
wherein the pre-trained language model has been trained using a training dataset on emotion data related to the specific emotion. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model trained in a generic manner with no additional details.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 10:
Step 1: A machine, as above.
Step 2A prong 1: See the rejection of Claim 9 above, which claim 10 depends on. Claim 10 further recites:
...calculating an intensity of emotion with respect to the specific emotion; and (A person mentally or with a pen and paper can assign a numerical intensity to an emotion expressed in text.)
...performing detailed emotion classification with respect to the specific emotion. (A person mentally or with a pen and paper can classify the detailed emotions expressed in text.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
wherein the pre-trained language model includes: a first language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model used as a tool to perform the abstract idea.)
a second language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic language model used as a tool to perform the abstract idea.)
Step 2B:
wherein the pre-trained language model includes: a first language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model used as a tool to perform the abstract idea.)
a second language model for... (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic language model used as a tool to perform the abstract idea.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 11:
Claim 11 is a device claim that recites substantially the same limitations as method claim 4. Therefore, claim 11 is rejected using the same rationale as claim 4.
Claim 12:
Step 1: The claim recites a non-transitory computer readable storage medium; therefore, it is directed to the statutory category of manufacture.
Step 2A prong 1: The claim recites the following abstract ideas:
generating clean data by removing errors included in the received text data; (A person mentally or with a pen and paper can read text data and strike out errors to produce clean data.)
detecting emotion data related to a specific emotion from the clean data and generating a training dataset including the detected emotion data; and (A person mentally or with a pen and paper can read the clean data, identify text related to a specific emotion and gather the identified text into a dataset.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
A non-transitory computer readable storage medium storing computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method for training language model, the method comprising: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic storage medium and processor used as tools to perform the abstract ideas.)
receiving text data; (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)). -- EN: Merely receiving text data, i.e., mere data gathering.)
training the language model using the training dataset. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. -- EN: Generic training of a generic language model with no additional details.)
Step 2B:
A non-transitory computer readable storage medium storing computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method for training language model, the method comprising: (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic storage medium and processor used as tools to perform the abstract ideas.)
receiving text data; (MPEP 2106.05(d)(II) indicates that merely gathering data is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
training the language model using the training dataset. (This additional element recites a mere instruction to apply an exception with a recitation of the words "apply it" (or an equivalent) as identified in MPEP 2106.05(f), and does not provide integration into a practical application. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. -- EN: Generic training of a generic language model with no additional details.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 13:
Claim 13 is a computer readable storage medium claim that recites substantially the same limitations as method claim 2. Therefore, claim 13 is rejected using the same rationale as claim 2.
Claim 14:
Claim 14 is a computer readable storage medium claim that recites substantially the same limitations as method claim 3. Therefore, claim 14 is rejected using the same rationale as claim 3.
Claim 15:
Claim 15 is a computer readable storage medium claim that recites substantially the same limitations as method claim 4. Therefore, claim 15 is rejected using the same rationale as claim 4.
Claim 16:
Claim 16 is a computer readable storage medium claim that recites substantially the same limitations as method claim 5. Therefore, claim 16 is rejected using the same rationale as claim 5.
Claim 17:
Claim 17 is a computer readable storage medium claim that recites substantially the same limitations as method claim 6. Therefore, claim 17 is rejected using the same rationale as claim 6.
Claim 18:
Claim 18 is a computer readable storage medium claim that recites substantially the same limitations as method claim 7. Therefore, claim 18 is rejected using the same rationale as claim 7.
Claim 19:
Claim 19 is a computer readable storage medium claim that recites substantially the same limitations as method claim 8. Therefore, claim 19 is rejected using the same rationale as claim 8.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claim(s) 1 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kabir et al., "DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities," (hereinafter Kabir).
Regarding claim 1, Kabir teaches:
A method for training a language model performed by a device for training the language model, the method comprising (Abstract, p. 1, "we present a new dataset of 40191 tweets labeled by expert annotators ... setting strong baseline results using attention-based models like BERT and DistilBERT" -- EN: fine-tuning the BERT and DistilBERT models on the dataset corresponds to the training of a language model; Section 6.1, p. 11, "The experiments were conducted in a computationally limited environment with a comparatively smaller batch size and fine-tuned only for 10 epochs." -- EN: the computing environment in which the fine-tuning is run denotes the device for training the language model):
receiving text data (Section 4, p. 5, "TWINT6 was used to collect tweets from Twitter for this study."; Section 4.1, p. 5, "Based on the final keyword list, a total of 344657 tweets were collected.");
generating clean data by removing errors included in the received text data (Section 4.2, p. 5, "From the collected samples, tweets that were posted in English were only preserved for annotation. Tweets with less than eight words were discarded as they might not contain enough context. Any tweets containing mentions (@) or hashtags (#), as well as retweets, were also discarded since they could violate the privacy of the users mentioned." -- EN: discarding the non-English, too-short, mentions, hashtags and retweeted tweets from the collected tweets corresponds to removing errors included in the received text data, and the tweets that remain denote the clean data; Fig. 1, p. 6, "random selection, filtering, and cleaning" -- EN: the cleaning applied to the raw tweets before the 44100 tweets are chosen denotes the generating of clean data; Section 4.3, p. 7, "1399 data samples were removed from the dataset because they were damaged (i.e., tweet text or tweet ID was changed) during the annotation process");
detecting emotion data related to a specific emotion from the clean data (Section 4.2.1, p. 6, "The task of the annotators was to label the tweets as one of the four classes, i.e., non-depressed, mildly depressed, moderately depressed, and severely depressed tweets."; Section 3, p. 3, "a user posting a tweet on social networking site Twitter is considered to be depressed if the tweet depicts behaviors portraying symptoms of depression" -- EN: the tweets depicting symptoms of depression denote the emotion data related to the specific emotion; Section 4.3, p. 7, "If at least two of the three annotators agreed on the label of a tweet, the matched annotation was accepted as the final label." -- EN: this denotes detecting the emotion data, each cleaned tweet's final label being set from the annotators' labels) and generating a training dataset including the detected emotion data (Section 4.3, p. 7, "The final dataset comprises a total of 40191 tweets along with their tweet_id, replies_count, retweets_count, likes_count, target, label and confidence_score." -- EN: the final dataset of labeled tweets denotes the training dataset; Abstract, p. 1, "Each tweet is labeled as ‘non-depressed’ or ‘depressed’. Moreover, three severity levels are considered for ‘depressed’ tweets: (1) mild, (2) moderate, and (3) severe." -- EN: the tweets labeled depressed, which are the detected emotion data, are included in the dataset); and
training the language model using the training dataset (Section 5.2.3, p. 10, "Both of these models were fine-tuned in a supervised manner for 10 epochs with a training batch size of 16 on the proposed dataset to predict the severity of depression from tweets"; Section 5.2.3, p. 10, "Fine-tuning the pre-trained model weights in a task specific manner with respect to the tweet texts and their annotated labels is necessary to improve the classification performance" -- EN: the tweet texts and their annotated labels are the training dataset used in the training).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kabir by Kabir et al., "DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities," (hereinafter Kabir).
Regarding claim 12, Kabir teaches:
… a method for training language model, the method comprising (Abstract, p. 1, "we present a new dataset of 40191 tweets labeled by expert annotators ... setting strong baseline results using attention-based models like BERT and DistilBERT" -- EN: fine-tuning the BERT and DistilBERT models on the dataset corresponds to the training of a language model):
receiving text data (Section 4, p. 5, "TWINT⁶ was used to collect tweets from Twitter for this study."; Section 4.1, p. 5, "Based on the final keyword list, a total of 344657 tweets were collected.");
generating clean data by removing errors included in the received text data (Section 4.2, p. 5, "From the collected samples, tweets that were posted in English were only preserved for annotation. Tweets with less than eight words were discarded as they might not contain enough context. Any tweets containing mentions (@) or hashtags (#), as well as retweets, were also discarded since they could violate the privacy of the users mentioned." -- EN: discarding the non-English, too-short, mentions, hashtags and retweeted tweets from the collected tweets corresponds to removing errors included in the received text data, and the tweets that remain denote the clean data; Fig. 1, p. 6, "random selection, filtering, and cleaning" -- EN: the cleaning applied to the raw tweets before the 44100 tweets are chosen denotes the generating of clean data; Section 4.3, p. 7, "1399 data samples were removed from the dataset because they were damaged (i.e., tweet text or tweet ID was changed) during the annotation process");
detecting emotion data related to a specific emotion from the clean data (Section 4.2.1, p. 6, "The task of the annotators was to label the tweets as one of the four classes, i.e., non-depressed, mildly depressed, moderately depressed, and severely depressed tweets."; Section 3, p. 3, "a user posting a tweet on social networking site Twitter is considered to be depressed if the tweet depicts behaviors portraying symptoms of depression" -- EN: the tweets depicting symptoms of depression denote the emotion data related to the specific emotion; Section 4.3, p. 7, "If at least two of the three annotators agreed on the label of a tweet, the matched annotation was accepted as the final label." -- EN: this denotes detecting the emotion data, each cleaned tweet's final label being set from the annotators' labels) and generating a training dataset including the detected emotion data (Section 4.3, p. 7, "The final dataset comprises a total of 40191 tweets along with their tweet_id, replies_count, retweets_count, likes_count, target, label and confidence_score." -- EN: the final dataset of labeled tweets denotes the training dataset; Abstract, p. 1, "Each tweet is labeled as ‘non-depressed’ or ‘depressed’. Moreover, three severity levels are considered for ‘depressed’ tweets: (1) mild, (2) moderate, and (3) severe." -- EN: the tweets labeled depressed, which are the detected emotion data, are included in the dataset); and
training the language model using the training dataset (Section 5.2.3, p. 10, "Both of these models were fine-tuned in a supervised manner for 10 epochs with a training batch size of 16 on the proposed dataset to predict the severity of depression from tweets"; Section 5.2.3, p. 10, "Fine-tuning the pre-trained model weights in a task specific manner with respect to the tweet texts and their annotated labels is necessary to improve the classification performance" -- EN: the tweet texts and their annotated labels are the training dataset used in the training).
Kabir does not explicitly teach:
A non-transitory computer readable storage medium storing computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform [a method for training language model, the method comprising:]
(EN: Bracketed portions added for context)
However, Kabir suggests:
A non-transitory computer readable storage medium storing computer executable instructions (Section 5.1, p. 9, "the overall performance boost comes at the price of higher computational power and memory needs for both training and inference, rendering them unsuitable for use on the edge devices, such as smartphones" -- EN: the memory that holds the BERT model for training and inference suggests the non-transitory storage medium, the training run on the device being computer-executable instructions held in that memory; Section 5.2.3, p. 10, "Base-uncased⁹ versions of the pre-trained models were implemented for fine-tuning with a total of 768 hidden output states." -- EN: this suggests the stored instructions, the implementation that fine-tunes the models being program code held in the device's memory), wherein the instructions, when executed by a processor, cause the processor to perform [a method for training language model, the method comprising:] (Section 5.1, p. 9, "Implementing a system that can detect the severity of depression from social media texts on devices with limited computational power may be difficult due to the high parameter count of BERT" -- EN: the computational power of the device on which the system is implemented suggests the processor; Section 6.1, p. 11, "The experiments were conducted in a computationally limited environment with a comparatively smaller batch size and fine-tuned only for 10 epochs." -- EN: the computing environment in which the fine-tuning is run suggests the processor that performs the training method; Section 4, p. 5, "TWINT⁶ was used to collect tweets from Twitter for this study." -- EN: this suggests the processor receiving the text data, TWINT being a software tool run by a processor; Section 4.2, p. 5, "From the collected samples, tweets that were posted in English were only preserved for annotation." -- EN: this suggests the processor generating the clean data, filtering the 344657 collected tweets being an operation that one of ordinary skill in the art would understand to be run by a processor; Section 4.2.2, p. 7, "Once the annotation process was finished, all the datafiles were merged and the control samples were removed from the dataset." -- EN: this suggests the processor detecting the emotion data and generating the training dataset as mapped above, merging the electronic datafiles and setting each tweet's final label being operations that one of ordinary skill in the art would understand to be run by a processor; Section 5.2.3, p. 10, "Both of these models were fine-tuned in a supervised manner for 10 epochs with a training batch size of 16 on the proposed dataset" -- EN: this suggests the processor training the language model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include, in the computing environment in which Kabir fine-tunes the models, a non-transitory computer readable storage medium holding the training method as computer executable instructions that a processor executes, as suggested by Kabir, in order to carry out the method on a programmed computing device (Section 5.1, p. 9 and Section 6.1, p. 11).
The motivation for doing so would be to allow Kabir's training method to be carried out by general-purpose computing hardware because the collection of the tweets with TWINT, the filtering of the collected tweets, the setting of each tweet's final label from the annotators' labels, and the fine-tuning of BERT and DistilBERT are embodied as computer executable instructions stored on the medium and executed by the processor, as taught by Kabir where the benefit is disclosed (Section 5.1, p. 9 and Section 6.1, p. 11).
Claim(s) 2, 9, 10, and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kabir in view of Zhang et al., "Symptom Identification for Interpretable Detection of Multiple Mental Disorders," (hereinafter Zhang).
Regarding claim 2, Kabir teaches all the limitations of claim 1, Kabir further teaches:
wherein the language model includes:
a first language model for calculating an intensity of emotion with respect to the specific emotion (Kabir Section 5.2.3, p. 10, "Both of these models were fine-tuned in a supervised manner for 10 epochs with a training batch size of 16 on the proposed dataset to predict the severity of depression from tweets" -- EN: the fine-tuned BERT model that predicts the severity of depression denotes the first language model, and the predicted severity denotes the intensity of emotion with the specific emotion being depression; Section 3, p. 4, "The frequency of these symptoms can help classify the severity of depression as none, mild, moderate, and severe conditions." -- EN: the graded levels none, mild, moderate and severe are degrees of depression; Fig. 4, p. 11, "Predicted Severity of Depression {1: Mildly Depressed}" -- EN: the classifier's output of the severity level for a tweet corresponds to calculating the intensity of emotion); and
Kabir does not explicitly teach:
a second language model for performing detailed emotion classification with respect to the specific emotion.
However, Zhang teaches:
a second language model for performing detailed emotion classification with respect to the specific emotion (Section 4.1, p. 6, "Overall, we use a BERT-based encoder (Devlin et al., 2018) with a linear layer on top of the representation of [CLS] to predict the probabilities of all symptoms with a sigmoid activation" -- EN: the BERT-based symptom relevance model denotes the second language model; Section 4.1, p. 5, "The task of relevance judgment can be viewed as a multi-label classification problem, where we need to predict for each symptom if it is relevant to the sentence." -- EN: this denotes predicting the detailed emotion classification; Section 3.1, p. 3, "The final knowledge graph (Figure 2) has 45 nodes (7 diseases and 38 symptoms), and 162 edges." -- EN: the symptoms linked to the Depression node of Figure 2, among them Depressed mood, Loss of interest or motivation, Worthless and guilty and Suicidal ideas, denote the detailed emotions with respect to depression; Section 1, p. 2, "We establish our annotation target (symptom classes) mainly based on the diagnostic criterions from DSM-5, with symptom descriptions on clinical questionnaires as supplementary." -- EN: the symptom classes are the DSM-5 diagnostic symptoms of the diseases).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include a second BERT model that reads the same tweet and predicts which DSM-5 symptoms of depression the tweet expresses, as taught by Zhang, in order to add a symptom-level output next to Kabir's severity level (Section 1, p. 1 and Section 5.5, p. 8).
The motivation for doing so would be to improve the accuracy and the interpretability of Kabir's depression detection because the second model tells the user which DSM-5 symptoms the tweet shows, as taught by Zhang where the benefit is disclosed (Abstract, p. 1; Section 1, p. 1; and Section 5.4, p. 8).
Regarding claim 9, Kabir teaches:
A device for emotion diagnosis, the device comprising (Section 5.1, p. 9, "Implementing a system that can detect the severity of depression from social media texts on devices with limited computational power may be difficult due to the high parameter count of BERT" -- EN: the device on which the severity detection system is implemented denotes the device for emotion diagnosis; Section 7, p. 13, "This work introduced a new typology for diagnosing depression severities from social media texts" -- EN: detecting the severity of depression from a text corresponds to emotion diagnosis):
a memory configured to store a pre-trained language model and one or more instructions for executing the pre-trained language model; and a processor configured to execute the one or more instructions stored in the memory (Section 5.1, p. 9, "the overall performance boost comes at the price of higher computational power and memory needs for both training and inference, rendering them unsuitable for use on the edge devices, such as smartphones" -- EN: the memory of the device holding the BERT model for inference denotes the memory storing the pre-trained language model and the instructions for executing it, and the computational power of the device that runs the inference denotes the processor executing those instructions, a BERT model implemented on a device being instructions held in its memory and run by its processor; Section 5.1, p. 9, "both BERT and DistilBERT are pre-trained using a large amount of data from English Wikipedia8 and Toronto Book Corpus" -- EN: BERT and DistilBERT denote the pre-trained language model),
wherein the instructions, when executed by the processor, cause the processor to:
receive text data for diagnosing an emotion (Fig. 4, p. 11, "i feel so tired, empty, drained and unmotivated. pls I can't even focus on my schoolworks. I just wanna cry so bad" -- EN: the tweet text fed to the fine-tuned classifier denotes the text data received for diagnosing depression), and
output an intensity of emotion … with respect to a specific emotion from the text data using the language model (Fig. 4, p. 11, "Fine-tuned BERT-based Classifier ... Predicted Severity of Depression {1: Mildly Depressed}" -- EN: the predicted severity level output by the fine-tuned BERT classifier denotes the intensity of emotion with respect to depression; Section 5.2.3, p. 10, "Both of these models were fine-tuned in a supervised manner for 10 epochs with a training batch size of 16 on the proposed dataset to predict the severity of depression from tweets" -- EN: predicting the severity from the tweet with the fine-tuned model corresponds to outputting the intensity from the text data using the language model),
wherein the pre-trained language model has been trained using a training dataset on emotion data related to the specific emotion (Section 5.2.3, p. 10, "Fine-tuning the pre-trained model weights in a task specific manner with respect to the tweet texts and their annotated labels is necessary to improve the classification performance" -- EN: fine-tuning the pre-trained BERT on the tweet texts and their labels corresponds to the pre-trained language model having been trained using the training dataset; Abstract, p. 1, "Each tweet is labeled as ‘non-depressed’ or ‘depressed’. Moreover, three severity levels are considered for ‘depressed’ tweets: (1) mild, (2) moderate, and (3) severe." -- EN: the tweets labeled for depression denote the emotion data related to the specific emotion).
Kabir does not explicitly teach:
[output an intensity of emotion] and classified detailed emotions [with respect to a specific emotion from the text data using the language model.]
(EN: Bracketed portions added for context)
However, Zhang teaches:
[output an intensity of emotion] and classified detailed emotions [with respect to a specific emotion from the text data using the language model.] (Section 4.3, p. 6, "First, we utilize the predicted probabilities from relevance judgment models trained on PsySym as the extracted symptom features." -- EN: the symptom probabilities that the relevance judgment model predicts for a post denote the classified detailed emotions output from the text data using the model; Section 5.5, p. 8, "The binarized prediction of symptoms of our model, and the disease-symptom relations from our knowledge graph (derived from clinical manuals) can help achieve this." -- EN: the binarized symptom predictions are the classified detailed emotions output for the user's posts; Table 3, p. 8, "I have a problem with hoarding. ... Obsession" -- EN: the symptom predicted for the post is an output of a classified detailed emotion; Section 3.1, p. 3, "The final knowledge graph (Figure 2) has 45 nodes (7 diseases and 38 symptoms), and 162 edges." -- EN: the symptoms linked to the Depression node of Figure 2 denote the detailed emotions with respect to depression).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include a second BERT model, stored and run on the device next to Kabir's severity classifier, that predicts which DSM-5 symptoms of depression the input tweet expresses and outputs them with the severity level, as taught by Zhang, in order to add a symptom-level output next to Kabir's severity level (Section 1, p. 1 and Section 5.5, p. 8).
The motivation for doing so would be to improve the accuracy and the interpretability of Kabir's depression detection because the second model tells the user which DSM-5 symptoms the tweet shows, as taught by Zhang where the benefit is disclosed (Abstract, p. 1; Section 1, p. 1; and Section 5.4, p. 8).
Regarding claim 10, it is an emotion diagnosis device claim of independent claim 9 and recites substantially the same limitations as claim 2, and is therefore rejected under the same rationale as claim 2.
Regarding claim 13, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 2, and is therefore rejected under the same rationale as claim 2.
Claim(s) 3 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kabir in view of Almerekhi et al., "Are These Comments Triggering? Predicting Triggers of Toxicity in Online Discussions," (hereinafter Almerekhi).
Regarding claim 3, Kabir teaches all the limitations of claim 1 including “wherein the generating clean data”.
Almerekhi teaches:
includes:
removing missing values from the text data (Section 3, p. 3035, "After collecting the comments, we removed the comments shown as deleted as well as comments that were removed by moderators." -- EN: a comment shown as deleted or removed carries no text, and removing those comments from the collected comments corresponds to removing missing values from the text data; Section 3, p. 3035, "the time stamp, b) ID of the comment, c) ID of the comment’s parent, and d) the comment text." -- EN: the retrieval ensures that every comment object carries its ID, its parent ID and its text, which are the fields the later check relies on); and
removing errors from the text data by performing a referential integrity check on data from which the missing values have been removed (Section 3, p. 3035, "Then, we constructed discussion threads from the comments using the ID and parent ID of the corresponding comment." -- EN: this denotes the referential integrity check, each comment's parent ID being matched against the comment IDs, performed after the deleted and removed comments were taken out; Section 3, p. 3035, "Following this procedure, we removed any comments that did not belong to a thread (i.e., parent comments with no children)." -- EN: this denotes removing errors from the text data, the comments that fail the ID and parent ID check being the errors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include, for text collected from a discussion forum, a cleaning step that first drops the comments shown as deleted or removed and then matches each comment's parent ID against the comment IDs and drops the comments that fail that check, as taught by Almerekhi, in order to clean the collected comments before annotation (Section 3, p. 3035). Kabir itself suggests collecting similar things from other social media and discussion forums (Kabir Section 7, p. 14).
The motivation for doing so would be to improve the quality of the corpus on which Kabir's classifier is trained because comments with no text and comments that fail the ID and parent ID check are removed before annotation, as taught by Almerekhi where the benefit is disclosed (Section 3, p. 3035).
Regarding claim 14, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 3, and is therefore rejected under the same rationale as claim 3.
Claim(s) 4, 5, 7, 8, 11, 15, 16, 18, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kabir in view of Zhang in view of Mohammad et al., "WASSA-2017 Shared Task on Emotion Intensity," (hereinafter Mohammad).
Regarding claim 4, Kabir teaches all the limitations of claim 1 including “wherein the training dataset”.
Zhang further teaches:
includes:
a second training dataset including emotion diagnosis information (Abstract, p. 1, "PsySym is annotated according to a knowledge graph of the 38 symptom classes related to 7 mental diseases complied from established clinical manuals and scales" -- EN: the PsySym corpus denotes the second training dataset, and its symptom labels drawn from the clinical manuals denote the emotion diagnosis information, in the combination the second training dataset being Kabir's tweets labeled with these symptoms in the same way; Section 1, p. 2, "We establish our annotation target (symptom classes) mainly based on the diagnostic criterions from DSM-5, with symptom descriptions on clinical questionnaires as supplementary." -- EN: the symptom classes are the DSM-5 diagnostic criteria of the diseases, which is diagnosis information for depression; Section 3.3, p. 5, "Finally, we recruited 31 volunteers contributed valid annotations for 8,554 sentences." -- EN: the annotated sentences carry the symptom labels; Section 3.4, p. 5, "We split the dataset into training/validation/testing set by 5:1:4" -- EN: the training split of PsySym is a training dataset).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include, in Kabir's training dataset, a second training dataset of Kabir's tweets labeled with the DSM-5 symptoms of depression, as taught by Zhang, in order to train the language model to identify which symptoms a text expresses (Abstract, p. 1 and Section 4.1, p. 5).
The motivation for doing so would be to improve the interpretability of Kabir's depression detection because the language model trained on the symptom-labeled tweets can report which DSM-5 symptoms a tweet shows, and symptoms are the bases psychiatrists use to make a diagnosis, as taught by Zhang where the benefit is disclosed (Abstract, p. 1; Section 1, p. 1; and Section 5.5, p. 8).
Kabir in view of Zhang do not explicitly teach:
a first training dataset including a collection of overall rankings of respondents; and
However, Mohammad teaches:
a first training dataset including a collection of overall rankings of respondents; and (Abstract, p. 1, "We create the first datasets of tweets annotated for anger, fear, joy, and sadness intensities using a technique called best–worst scaling (BWS). We show that the annotations lead to reliable fine-grained intensity scores (rankings of tweets by intensity)." -- EN: the Tweet Emotion Intensity Dataset denotes the first training dataset, and the rankings of tweets by intensity derived from the annotators' choices denote the collection of overall rankings of respondents, in the combination the first training dataset being Kabir's tweets scored for depression intensity in the same way; Section 2.2, p. 3, "Annotators are given n items (an n-tuple, where n > 1 and commonly n = 4). They are asked which item is the best (highest in terms of the property of interest) and which is the worst (lowest in terms of the property of interest)." -- EN: the annotators denote the respondents; Section 3.1, p. 5, "For each item t, the score is the percentage of times the t was chosen as having the most intensity minus the percentage of times t was chosen as having the least intensity." -- EN: the score aggregated over all annotators' choices for an item denotes its overall ranking; Section 4.2, p. 6, "The Tweet Emotion Intensity Dataset is partitioned into training, development, and test sets for machine learning experiments" -- EN: the training partition is a training dataset)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir in view of Zhang to include, as the training dataset of Kabir's severity classifier, tweets scored for depression intensity by best-worst scaling, each score being the percentage of 4-tuples in which the annotators chose the tweet as the most intense minus the percentage in which they chose it as the least intense, as taught by Mohammad, in order to obtain a fine-grained intensity score for every tweet from comparative judgments (Section 2.2, p. 3 and Section 3.1, p. 5).
The motivation for doing so would be to improve the reliability of the intensity labels on which Kabir's classifier is trained because each annotator only compares four tweets and picks the most and the least intense, which avoids the inconsistent direct scores and the scale region bias of rating scales, and the picks aggregated over eight 4-tuples per tweet give fine-grained scores with high split-half reliability, as taught by Mohammad where the benefit is disclosed (Section 2.1, p. 3; Section 2.2, p. 3; and Section 3.2, p. 5).
Regarding claim 5, Kabir in view of Zhang in view of Mohammad teaches all the limitations of claims 1 and 4, Kabir further teaches:
wherein the generating the first training dataset includes:
extracting data including keywords related to the specific emotion from the clean data (Kabir Section 4.1, p. 5, "Based on the final keyword list, a total of 344657 tweets were collected." -- EN: every collected tweet contains one of the 88 depression-related keywords; Section 4.2, p. 5, "From the collected samples, tweets that were posted in English were only preserved for annotation." -- EN: the preserved tweets denote the clean data, each still containing the keywords; Section 4.2, p. 5, "Finally, 44100 tweets were randomly chosen from the remaining tweets for annotation." -- EN: choosing the 44100 tweets from the remaining cleaned tweets denotes extracting data from the clean data, and the chosen tweets include the keywords related to depression, the claim reciting that the extracted data includes the keywords, which in Kabir it does. Therefore, the claim is taught under BRI.); and
obtaining data related to symptoms related to the specific emotion as emotion data from the extracted data (Kabir Section 3, p. 3, "a user posting a tweet on social networking site Twitter is considered to be depressed if the tweet depicts behaviors portraying symptoms of depression" -- EN: the tweets depicting symptoms of depression denote the data related to symptoms related to the specific emotion; Section 3.2, p. 5, "A mildly depressed tweet may contain symptoms of hopelessness, feelings of guilt or despair, difficulties concentrating at work, a loss of interest in activities" -- EN: the symptoms define the depressed classes into which the extracted tweets are labeled; Section 4.2.1, p. 6, "The task of the annotators was to label the tweets as one of the four classes, i.e., non-depressed, mildly depressed, moderately depressed, and severely depressed tweets." -- EN: labeling the chosen tweets as depressed corresponds to obtaining, from the extracted data, the symptom-related tweets as emotion data).
Regarding claim 7, Kabir in view of Zhang in view of Mohammad teaches all the limitations of claims 1, 4, and 5, Mohammad further teaches:
-- EN: claim 7 recites "The method of claim 9", and claim 9 is a device claim in which the first training dataset has no antecedent basis. Claim 7 is examined as best understood to depend on claim 5, in parallel with claim 18, which depends on claim 16.
further comprising, after the generating the first training dataset (Mohammad Section 3.1, p. 4, "2 × N (where N is the number of tweets in the emotion set) distinct 4-tuples were randomly generated in such a manner that each item is seen in eight different 4-tuples" -- EN: this denotes arranging an already collected set of tweets into 4-tuples for annotation, the set in the combination being Kabir's first training dataset generated as set out for claim 5; Section 3.1, p. 4, "The 4-tuples of tweets were uploaded on the crowdsourcing platform, CrowdFlower." -- EN: this denotes the annotation taking place after the first training dataset is generated), matching predetermined annotation information to the emotion data based on classification conditions preset for the the first training dataset (Mohammad Section 3.1, p. 4, "For each emotion, the annotators were presented with four tweets at a time (4-tuples) and asked to select the speakers of the tweets with the highest and lowest emotion intensity." -- EN: the annotators' selection of the most and the least intense tweet of each 4-tuple denotes the predetermined annotation information matched to the tweets, and the selection criteria of highest and lowest intensity denote the classification conditions preset for the dataset, the tweets in the combination being Kabir's emotion data obtained as set out for claim 5; Appendix 8.1, p. 15, "Q1. Which of the four speakers is likely to be the MOST fearful? ... Q2. Which of the four speakers is likely to be the LEAST fearful?" -- EN: the questionnaire's fixed questions are the preset conditions under which each tweet is annotated; Section 3.1, p. 5, "The BWS responses were translated into scores by a simple calculation (Orme, 2009; Flynn and Marley, 2014): For each item t, the score is the percentage of times the t was chosen as having the most intensity minus the percentage of times t was chosen as having the least intensity." -- EN: the responses attached to each tweet are the annotation information matched to the emotion data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir in view of Zhang to include, as the training dataset of Kabir's severity classifier, tweets scored for depression intensity by best-worst scaling, each score being the percentage of 4-tuples in which the annotators chose the tweet as the most intense minus the percentage in which they chose it as the least intense, as taught by Mohammad, in order to obtain a fine-grained intensity score for every tweet from comparative judgments (Section 2.2, p. 3 and Section 3.1, p. 5).
The motivation for doing so would be to improve the reliability of the intensity labels on which Kabir's classifier is trained because each annotator only compares four tweets and picks the most and the least intense, which avoids the inconsistent direct scores and the scale region bias of rating scales, and the picks aggregated over eight 4-tuples per tweet give fine-grained scores with high split-half reliability, as taught by Mohammad where the benefit is disclosed (Section 2.1, p. 3; Section 2.2, p. 3; and Section 3.2, p. 5).
Regarding claim 8, Kabir in view of Zhang in view of Mohammad teaches all the limitations of claims 1, 4, and 5, Kabir further teaches:
wherein the generating the second training dataset includes:
extracting similar data including similar words … based on preset base words related to the specific emotion (Kabir Section 4.1, p. 5, "Seed terms were generated from the keywords extracted from each of the symptoms of PHQ-9 questionnaire by collaborating with two professional psychologists." -- EN: the seed terms denote the preset base words related to depression; Section 4.1, p. 5, "For this study, the seed terms for each questionnaire of PHQ-9 were extended by WordNet, and the extended terms were handpicked afterward by the psychologist collaborators." -- EN: the terms WordNet supplies from the seed terms denote the similar words; Section 4.1, p. 5, "Based on the final keyword list, a total of 344657 tweets were collected." -- EN: the tweets collected for containing the extended terms denote the similar data including the similar words); and
obtaining the similar data … as emotion data (Kabir Section 4.2.1, p. 6, "The task of the annotators was to label the tweets as one of the four classes, i.e., non-depressed, mildly depressed, moderately depressed, and severely depressed tweets." -- EN: the collected tweets labeled as depressed are the similar data obtained as emotion data).
Kabir does not explicitly teach:
[extracting similar data including similar words] from the clean data [based on preset base words related to the specific emotion; and]
[obtaining the similar data] and daily data [as emotion data.]
(EN: Bracketed portions added for context)
However, Zhang teaches:
[extracting similar data including similar words] from the clean data [based on preset base words related to the specific emotion; and] (Appendix B, p. 13, "For each post, we conduct several preprocessing steps. First, they are split into sentences with blingfire6. We will then eliminate sentences like “[Removed]”." -- EN: the post sentences left after the preprocessing denote the clean data; Section 3.2, p. 3, "we use Sentence-BERT (Reimers and Gurevych, 2019) to encode the post sentences and the symptom descriptions in the KG into embeddings. Then we will calculate the cosine similarity between them, and estimate a sentence’s relevance to a symptom with its max similarity with all of the symptom’s sub-symptoms." -- EN: retrieving, from the preprocessed post sentences, the sentences most similar to the symptom descriptions corresponds to extracting similar data from the clean data based on preset words for the symptoms, Kabir teaching the similar words as mapped above)
[obtaining the similar data] and daily data [as emotion data.] (Section 3.4, p. 5, "we also collect such posts (referred to as control posts) for PsySym from the same candidate pool. We randomly sample posts whose author doesn’t have any post or comment in mental health related subreddits" -- EN: the control posts, everyday posts of authors with no mental health subreddit activity, denote the daily data; Section 4.1, p. 6, "When trained on PsySym with control posts, the especially unbalanced positive/negative ratio can make the training hard." -- EN: the symptom model is trained on the symptom sentences and the control posts together, which corresponds to obtaining both as emotion data)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir to include a second BERT model that reads the same tweet and predicts which DSM-5 symptoms of depression the tweet expresses, as taught by Zhang, in order to add a symptom-level output next to Kabir's severity level (Section 1, p. 1 and Section 5.5, p. 8).
The motivation for doing so would be to improve the accuracy and the interpretability of Kabir's depression detection because the second model tells the user which DSM-5 symptoms the tweet shows, as taught by Zhang where the benefit is disclosed (Abstract, p. 1; Section 1, p. 1; and Section 5.4, p. 8). The listed symptoms explain the severity level the way a clinician explains a diagnosis. The symptom scores also serve as input features that beat a plain BERT text classifier (Section 5.4, p. 8).
Regarding claim 11, it is an emotion diagnosis device claim of independent claim 9 and recites substantially the same limitations as claim 4 and is therefore rejected under the same rationale as claim 4.
Regarding claim 15, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 4, and is therefore rejected under the same rationale as claim 4.
Regarding claim 16, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 5, and is therefore rejected under the same rationale as claim 5.
Regarding claim 18, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 7, and is therefore rejected under the same rationale as claim 7.
Regarding claim 19, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 8, and is therefore rejected under the same rationale as claim 8.
Claim(s) 6 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kabir in view of Zhang in view of Mohammad in view of Nguyen et al., "Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires," (hereinafter Nguyen).
Regarding claim 6, Kabir in view of Zhang in view of Mohammad teaches all the limitations of claims 1, 4, and 5 including “wherein the generating the first training dataset includes” extracting data including keywords related to the specific emotion from the clean data, as set out above for claim 5.
-- EN: claim 6 recites "The method of claim 9", and claim 9 is a device claim. Claim 6 is examined as best understood to depend from claim 5, in parallel with claim 17, which depends from claim 16.
Kabir teaches:
extracting data including keywords … related to the specific emotion from the clean data (Kabir Section 4.1, p. 5, "Based on the final keyword list, a total of 344657 tweets were collected." -- EN: every collected tweet contains one of the 88 depression-related keywords; Section 4.2, p. 5, "From the collected samples, tweets that were posted in English were only preserved for annotation." -- EN: the preserved tweets denote the clean data, each still containing the keywords; Section 4.2, p. 5, "Finally, 44100 tweets were randomly chosen from the remaining tweets for annotation." -- EN: choosing the 44100 tweets from the remaining cleaned tweets denotes extracting data from the clean data, and the chosen tweets include the keywords related to depression, the claim reciting that the extracted data includes the keywords, which in Kabir it does. Therefore, the claim is taught under BRI.)
Kabir in view of Zhang in view of Mohammad do not explicitly teach:
[extracting data including keywords] and negative words [related to the specific emotion from the clean data] and obtaining the data as daily data
(EN: Bracketed portions added for context)
However, Nguyen teaches:
[extracting data including keywords] and negative words [related to the specific emotion from the clean data] (Section 4.2.1, p. 5, "Negation. Posts containing symptom patterns with negation words (e.g., “not”, “never”) preceding." -- EN: the negation words preceding the symptom patterns denote the negative words, and the posts holding both are the extracted data; Appendix C.1, p. 13, "To handle this situation, we removed posts containing positive patterns preceded by a negation word, such as (“not”, “never”, ’rarely”, etc.)." -- EN: identifying, among the matched posts, those whose pattern is preceded by a negation word corresponds to the extracting, Kabir teaching the keywords and the clean data as mapped for claim 5, Nguyen then removing these posts from the positive class) and obtaining the data as daily data (Section 4.2.2, p. 5, "We use five heuristics to identify and synthesize negative examples for each symptom:" -- EN: the negative examples, posts that do not express the symptom, denote the daily data; Section 4.2.2, p. 5, "Negation. Posts synthesized by negating symptom patterns in positive examples using hand-crafted mappings (e.g., “tired“ to “never tired“)." -- EN: this denotes negated symptom mentions used as negative examples. Nguyen makes these by negating positive examples instead of keeping the removed posts, and the modification below keeps the extracted posts themselves as the negative examples).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Kabir in view of Zhang in view of Mohammad to include, in the first training dataset, the tweets in which a depression keyword is preceded by a negation word, held as non-depressed examples instead of being removed or labeled depressed, as taught by Nguyen, in order to keep negated mentions of a symptom out of the depressed class and use them as hard negatives (Section 4.2.1, p. 5 and Section 4.2.2, p. 5).
The motivation for doing so would be to improve the accuracy of the classifier trained on the dataset because a negated symptom mention, a false positive if labeled depressed, instead teaches the classifier that the keyword alone does not mark the symptom, and hard negatives of that kind keep the model from performing only keyword matching, as taught by Nguyen where the benefit is disclosed (Section 4.7, p. 8; Appendix C.1, p. 13; and Appendix C.2, p. 13).
Regarding claim 17, it is a non-transitory computer readable storage medium claim of independent claim 12 and recites substantially the same limitations as claim 6, and is therefore rejected under the same rationale as claim 6.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAYMUR RAHMAN ALI whose telephone number is (571)272-0007. The examiner can normally be reached Mon-Fri. 9:30-6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAYMUR RAHMAN ALI/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights