DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 05/23/2024. Claims pending. Claims independent.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: (1) . Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The use of the term "WiMAX" in ¶ [0102], which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Claim Objections
Claims 7, 9, 17, and 19 are objected to because of the following informalities:
in Claim 7, line 1, "… wherein the operations comprise …" appears to be "… wherein the operations further comprise …";
in Claim 9, line 1, "… wherein the operations comprise …" appears to be "… wherein the operations further comprise …";
in Claim 17, line 1, "The method of claim 11, comprising …" appears to be "The method of claim 11, further comprising …";
in Claim 19, line 1, "The method of claim 17, comprising …" appears to be "The method of claim 11, further comprising …".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 11, and 20 recite the limitation "… identifying a plurality of data items associated with user-generated content; … each label describing a sentiment of user-generated content associated with a respective data item …" in lines 6-11, 2-7, and 4-9 respectively, which rendering these claims indefinite because it is unclear whether two instances of ".
Claims 2-10 and 12-19 are rejected for fully incorporating the deficiency of their respective base claims.
Claims 5 and 15 recite the limitation "… wherein the plurality of data items associated with user-generated content comprises …" in lines 1-2, which rendering these claims indefinite because ".
Claims 6 and 16 recite the limitation "…" in lines 1-2, which rendering these claims indefinite because ".
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Independent Claims 1, 11, and 20
Step 1: Claim 1 is a system claim, Claim 11 is a process claim , and Claim 20 is a claim for a machine-storage medium (excluding carrier waves and signal medium in ¶ [0101]). These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) recite(s) "identifying a plurality of data items associated with user-generated content ", "annotating a first subset of data items in the plurality of data items, the annotating of the first subset of data items comprising generating a first plurality of labels for the first subset of data items, each label describing a sentiment of user-generated content associated with a respective data item", and "annotating a second subset of data items in the plurality of data items, the annotating of the second subset of data items comprising generating a second plurality of labels for the second subset of data items" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) recite(s) additional elements/limitations of "a system" (Claim 1), "one or more hardware processors " (Claims 1 and 20), "machine-storage medium" (Claims 1 and 20), "using a first machine learning (ML) model", "training a second ML model based on the first plurality of labels generated for the first subset of data items", "using the trained second ML model", and "training a third ML model based on the second plurality of labels generated for the second subset of data items" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitations/elements of "using a first machine learning (ML) model", "training a second ML model based on the first plurality of labels generated for the first subset of data items", "using the trained second ML model", and "training a third ML model based on the second plurality of labels generated for the second subset of data items" are well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 2 and 12
Step 1: Claim 2 is a system claim and Claim 12 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "using a first machine learning (ML) model, wherein the first ML model comprises a large-scale Large Language Model having weights of more than 100 billion parameters" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "using a first machine learning (ML) model, wherein the first ML model comprises a large-scale Large Language Model having weights of more than 100 billion parameters" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 3 and 13
Step 1: Claim 3 is a system claim and Claim 13 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "training a second ML model based on the first plurality of labels generated for the first subset of data items, wherein the second ML model comprises a medium-scale Large Language Model having weights between 1 billion parameters and 100 billion parameters" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "training a second ML model based on the first plurality of labels generated for the first subset of data items, wherein the second ML model comprises a medium-scale Large Language Model having weights between 1 billion parameters and 100 billion parameters" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 4 and 14
Step 1: Claim 4 is a system claim and Claim 14 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) does/do not further recite(s) elements/limitations which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/algorithms/calculations.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "training a third ML model based on the second plurality of labels generated for the second subset of data items, wherein the third ML model comprises a small-scale Large Language Model having weights of less than 1 billion parameters" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "training a third ML model based on the second plurality of labels generated for the second subset of data items, wherein the third ML model comprises a small-scale Large Language Model having weights of less than 1 billion parameters" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 5 and 15
Step 1: Claim 5 is a system claim and Claim 15 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "identifying a plurality of data items associated with user-generated content, wherein the plurality of data items associated with user-generated content comprises one or more of a plurality of comments and a plurality of reviews " which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 6 and 16
Step 1: Claim 6 is a system claim and Claim 16 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "annotating a first subset of data items in the plurality of data items, the annotating of the first subset of data items comprising generating a first plurality of labels for the first subset of data items, each label describing a sentiment of user-generated content associated with a respective data item, wherein the sentiment of user-generated content corresponds to a model output value representing positive, negative, or neutral" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 7 and 17
Step 1: Claim 7 is a system claim and Claim 17 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "determining a confidence value based on the first plurality of labels for the first subset of data items" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "training the second ML model and the third ML model based on the confidence value" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "training the second ML model and the third ML model based on the confidence value" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 8 and 18
Step 1: Claim 8 is a system claim and Claim 18 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "determining a confidence value based on the first plurality of labels for the first subset of data items, wherein the confidence value represents an accuracy of annotation for the first subset of data items" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) does/do not further recite(s) additional elements/limitations.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 9 and 19
Step 1: Claim 9 is a system claim and Claim 19 is a process claim. These claims fall within at least one of the four categories of patent eligible subject matter.
Step 2A Prong 1: The claim(s) further recite(s) "configuring a model output probability based on the confidence value" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper") or mathematical concepts/calculations/algorithms.
Step 2A Prong 2: This judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "training the second ML model and the third ML model based on the model output probability" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not further include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element of "training the second ML model and the third ML model based on the model output probability" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a non-statutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based e-Terminal Disclaimer may be filled out completely online using web-screens. An e-Terminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about e-Terminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 3-4, 5-6, 11, 13-14, 16, and 20 are provisionally rejected on the ground of non-statutory double patenting as being unpatentable over Claims 1, 2-3, 10, 9, 11-13, 19, and 20 of co-pending Application No. 18/672,573 in view of Mirzadeh et al. ("Improved Knowledge Distillation via Teacher Assistant",arXiv:1902.03393v2, dec 17, 2019, pp. 1-11), hereinafter Mirzadeh.
Claims 1, 2-3, 10, 9, 11-13, 19, and 20 of co-pending Application No. 18/672,573 discloses all the limitations of Claims 1, 3-4, 5-6, 11, 13-14, 16, and 20 of the instance application except using three ML models instead of two ML models in the co-pending Application No. 18/672,573. Mirzadeh teaches in Abstract and Introduction in Page 1 with FIG. 1 that introduce multi-step knowledge distillation, which employs an intermediate-sized network (teacher assistant) (i.e., 2nd ML model) to bridge the gap between the student (i.e., 3rd ML model) and the teacher (i.e., 1st ML model); i.e., propose a new distillation framework called Teacher Assistant Knowledge Distillation (TAKD), which introduces intermediate models as teacher assistants (TAs) between the teacher and the student to fill in their gap (Figure 1). It would have been obvious to one of ordinary skill in the art to apply teaching of Mirzadeh for improving the student network performance in the case of extreme compression (Mirzadeh, Abstract in Page 1 and Teacher Assistant Knowledge Distillation (TAKD) in Pages 3-4). This is a provisional non-statutory double patenting rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-7, 9, 11-12, 14-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over BUBECK et al. (US 2025/0086471 A1, priority date: 09/11/2023), hereinafter BUBECK in view of Karamouzas et al. ("Neural Knowledge Transfer for Sentiment Analysis in Texts with Figurative Language", 2022 IEEE 32nd International Workshop on Machine Learning for Signal Processing (MLSP), Aug. 22-25, 2022, pp. 1-6), hereinafter Karamouzas.
Independent Claims 1, 11, and 20
BUBECK discloses a system (BUBECK, ¶¶ [0027] and [0037] with FIGS. 2-3: an operating environment 200 includes a server 230 and a computing device 220; the method 300 is performed by a computing device ( e.g., a server 230); ¶ [0069] with 500 in FIG. 5: a computing device 500; ¶ [0077] with 600 in FIG. 6: a system 600; ¶ [0087] with FIG. 7: a system) comprising: one or more hardware processors (BUBECK, ¶ [0027] with 220 in FIG. 2: a processor 222; ¶ [0038] with FIG. 3: the method 300 can be performed by gates or circuits associated with a processor, Application Specific Integrated Circuit (ASIC), a field programmable gate array FPGA), a system on chip (SOC), a Neural Processing Unit (NPU), or other hardware device; ¶¶ [0069] and [0073] with 502 in FIG. 5: at least one processing unit 502; ¶¶ [0084] and [0102] with 660/661 in FIG. 6: the processor 660/specifical purpose processor 661); and at least one machine-storage medium (BUBECK, ¶ [0027] with 224 in FIG.2: memory 224; ¶ [0038] with FIG. 3: a computer readable medium; ¶¶ [0069]-[0070] and [0075] with 504 and 510 in FIG. 5: a system memory 504; the non-removable storage device 510; computer readable media as used herein may include computer storage media; ¶ [0081] with 662 and 668 in FIG. 6: a non-volatile storage area 668 within the memory 662; ¶¶ [0104] and [0121]: a computer storage medium) for storing instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations (BUBECK, ¶¶ [0037]-[0038] with FIGS. 2-3: the method 300 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium; e.g., the set of computer-executable instructions stored on a storage device may cause a computing device ( e.g., a server 230) to perform the method 300; the method 300 can be performed by gates or circuits associated with a processor, Application Specific Integrated Circuit (ASIC), a field programmable gate array (FPGA), a system on chip (SOC), a Neural Processing Unit (NPU), or other hardware device; ¶ [0072] with FIG. 5: a number of program modules and data files may be stored in the system memory 504; while executing on the processing unit 502, the program modules 506 ( e.g., application 520) may perform processes including, but not limited to, the aspects, as described herein; ¶ [0081] with FIG. 6: one or more application programs 666 may be loaded into the memory 662 and run on or in association with the operating system 664; ¶ [0104]: methods may be partially implemented in software that can be stored on a storage medium, executed on programmed general purpose computer with the cooperation of a controller and memory, a special purpose computer, a microprocessor, or the like) comprising:
identifying a plurality of data items associated with (BUBECK, ¶¶ [0005]-[0007]: obtaining a general dataset, the general dataset including a plurality of general data; ¶¶ [0022] and [0029] with FIGS. 2 and 3A: a general dataset including a plurality of general data is obtained from various sources (e.g., web sources); the general dataset manager 242 is configured to obtain a general dataset from various sources (e.g., web sources); the general dataset includes a plurality of general data; ¶ [0040] with FIGS. 2 and 3A: at operation 304, the language model generator 240 obtains a general dataset from various sources (e.g., web sources); the general dataset includes a plurality of general data);
annotating, using a first machine learning (ML) model, a first subset of data items in the plurality of data items, the annotating of the first subset of data items comprising generating a first plurality of labels for the first subset of data items, each label describing (BUBECK, ¶¶ [0005]-[0007]: annotating a subset of the general dataset based on one or more classifier metrics indicative of a quality of the general dataset, the subset of the general dataset being representative of the general dataset; ¶¶ [0022] with FIG. 1: once the general dataset is obtained, a subset of the general dataset that represents the general dataset is selected and an annotated version of the subset of the general dataset is generated based on one or more classifier metrics indicative of a quality of the subset of the general dataset; a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; e.g., the generative transformer may be a language model (e.g., a large language model) or any generative machine learning model capable of annotating data with specific attributes or features; the one or more classifier metrics are specific attributes or features associated with the subset of the general dataset; the one or more classifier metrics include, but not limited to, factual knowledge, everyday knowledge, scientific knowledge, human behavior, toxicity, completeness, obscenity, obscurity, commonality, reasoning, promotional content, and/or unwanted content; each general data of the subset of the general dataset is annotated with a score for each classifier metric of the one or more classifier metrics using the generative transformer; ¶ [0029] with FIG. 2: the general dataset manager 242 is configured to select a subset of the general dataset that represents the general dataset and generate an annotated version of the subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; e.g., the generative transformer may be a language model ( e.g., a large language model) or any generative machine learning model capable of annotating data with specific attributes or features; the general dataset manager 242 is configured to annotate each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; ¶ [0041] with FIGS. 2 and 3A: at operation 306, the language model generator 240 annotates a subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a subset of the general dataset is selected to represent the general dataset; once the subset of the general dataset is selected, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; e.g., the generative transformer may be a language model (e.g., a large language model) or any generative machine learning model capable of annotating data with specific attributes or features; the language model generator 240 annotates each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; e.g., the score may be on a scale of one to ten with one being the lowest and ten being the highest);
training a second ML model based on the first plurality of labels generated for the first subset of data items (BUBECK, ¶¶ [0005]-[0007]: training a classifier based on the annotated subset of the general dataset and the one or more classifier metrics; ¶ [0023] with FIG. 1: upon annotating the subset of the general dataset, a classifier is trained using the annotated subset of the general dataset; ¶ [0030] with FIG. 2: upon annotating the subset of the general dataset, the general dataset manager 242 is configured to train a classifier using the annotated subset of the general dataset; ¶ [0042] with FIGS. 2 and 3A: upon annotating the subset of the general dataset, the language model generator 240 trains a classifier using the annotated subset of the general dataset as indicated in operation 308) ;
annotating, using the trained second ML model, a second subset of data items in the plurality of data items, the annotating of the second subset of data items comprising generating a second plurality of labels for the second subset of data items (BUBECK, ¶¶ [0005]-[0007]: analyzing each general data of the general dataset to determine a score for each of the one or more classifier metrics associated with the respective general data using the trained classifier, generating a filtered general dataset by filtering the general dataset based on one or more filters, the one or more filters indicative of threshold scores for corresponding classifier metrics; generating a synthetic dataset for refining the small language model; ¶¶ [0023]-[0024] with FIG. 1: the trained classifier is configured to predict quality of data based on the one or more classifier metrics; the quality of data may be represented by a score for each classifier metric; once the classifier is trained, each general data of the general dataset is analyzed to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, one or more filters for filtering the general dataset are generated; each filter indicates a threshold score for a respective classifier metric; in other words, the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; ¶¶ [0025]-[0026] with FIG. 1: the second training phase 130 involves generating a synthetic dataset and training the small language model with the synthetic dataset; the synthetic dataset is "textbook- like" data that is created for the purposed of teaching common sense reasoning and general knowledge of the world (e.g., science, daily activities, theory of mind, etc.); e.g., specific topics may be selected to seed the generation of the synthetic data; one or more deficit skills in the small language model are identified; the term deficit skill means any skill or topic that may boost in the capability of the small language model; once the one or more deficit skills are identified, one or more data formats to address the one or more deficit skills are determined and one or more prompts are obtained to create the one or more data formats; additionally, sources of randomization and diversity may be injected into the one or more prompts; based on the one or more prompts, the synthetic dataset is generated using a generative transformer; the generative transformer may be any generative machine learning model (e.g., a large language model) capable of generating data based on a prompt; the synthetic dataset is designed to provide and boost certain skills in the small language model; ¶¶ [0030]-[0032] with FIG. 2: the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; once the classifier is trained, the general dataset manager 242 is further configured to analyze each general data of the general dataset; specifically, the general dataset manager 242 is configured to analyze each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the general dataset manager 242 is configured to generate one or more filters for filtering the general dataset; each filter indicates a threshold score for a respective classifier metric; the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the general dataset manager 242 is configured to generate the filtered general dataset; specifically, the general dataset manager 242 is configured to generate the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; each filter is associated with a respective classifier metric; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; ¶¶ [0033]-[0035] with FIG. 2: the synthetic dataset manager 244 is configured to generate a synthetic dataset to be used during the second training phase of the small language model; the synthetic dataset manager 244 is configured to identify one or more deficit skills in the small language model; the synthetic dataset manager 244 is further configured to determine one or more data formats to address the one or more deficit skills and obtain one or more prompts to create the one or more data formats; the synthetic dataset manager 244 is configured to inject sources of randomization and diversity into the one or more prompts; the synthetic dataset manager 244 is configured to generates the synthetic dataset based on the one or more prompts using a generative transformer; e.g., the generative transformer may be any generative machine learning model (e.g., a large language model) capable of generating data based on a prompt; in aspects, the synthetic dataset is designed to provide and boost certain skills in the small language model; to do so, the generative transformer is prompted to synthesize textbook-style content concerning the skills that require to be boosted; e.g., detailed lists of topics, subtopics, keywords, as well as content, related to those skills, that was filtered from other sources such as web pages may be provided and used as a seed to the generative transformer; the seed is shuffled and combined with styles, use-cases, user roles, and other sources of randomness that give rise to diverse ways of presenting the content; all of these may be combined as an input into the generative transformer's prompt in different random combinations; output of the generative transformer may then be integrated to generate the synthetic dataset; ¶¶ [0042]-[0045] with FIGS. 2 and 3A: the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; subsequent to training the classifier, the language model generator 240 analyzes each general data of the general dataset as indicated in operation 310; the language model generator 240 analyzes each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the language model generator 240 generates one or more filters for filtering the general dataset as indicated in operation 312; each filter indicates a threshold score for a respective classifier metric; i.e., the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the language model generator 240 generates a filtered general dataset as indicated in operation 314; the language model generator 240 generates the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; each filter is associated with a respective classifier metric; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; ¶¶ [0047]-[0052] with FIGS. 2 and 3B: one or more operations from the second training phase for generating the synthetic dataset ( e.g., 318-328) may be performed in parallel with one or more operations from the first training phase for generating the filtered general dataset; at operation 318, the language model generator 240 generates a synthetic dataset for refining the small language model; to do so, at operation 320, the language model generator 240 identifies one or more deficit skills in the small language model; at operation 322, the language model generator 240 determines one or more data formats to address the one or more deficit skills; e.g., the data format may be texts, graphs, and/or images; at operation 324, the language model generator 240 obtains one or more prompts to create the one or more data formats; at operation 326, sources of randomization and diversity is injected in the one or more prompts; the generative transformer is prompted to synthesize textbook-style content concerning the deficit skills that require to be boosted; e.g., detailed lists of topics, subtopics, keywords, as well as content, related to those skills, that was filtered from other sources such as web pages may be provided and used as a seed to the generative transformer; at operation 328, the language model generator 240 generates the synthetic dataset based on the one or more prompts using a generative transformer; e.g., the generative transformer may be any generative machine learning model ( e.g., a large language model) capable of generating data based on a prompt; in aspects, the synthetic dataset is designed to provide and boost certain skills in the small language model); and
training a third ML model based on the second plurality of labels generated for the second subset of data items (BUBECK, ¶¶ [0005]-[0007]: training the small language model with the filtered general dataset; subsequent to training the small language model with the filtered general dataset, training the small language model with the synthetic dataset; ¶¶ [0023]-[0024] with FIG. 1: the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; the filtered general dataset is used to train the small language model in the first training phase 120; ¶¶ [0025]-[0026] with FIG. 1: the second training phase 130 involves generating a synthetic dataset and training the small language model with the synthetic dataset; the filtered general dataset is used to train the small language model in the second training phase 130; ¶¶ [0031]-[0032] with FIG. 2: the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; ¶ [0036] with FIG. 1: the small language model manager 246 is configured to perform the two-phase training to train a small language model; specifically, the small language model manager 246 is configured to train the small language model with the filtered general dataset during the first training phase and with the synthetic dataset during the second training phase; by curating the training data for the small language model, the small language model manager 246 is configured to train the small language model with data that satisfies certain quality suitable for certain application; e.g., the small language model manager 246 may reduce toxic and biased content among the general dataset and the synthetic dataset; i.e., two separate sources of training data in used over two-phase training to train the small language model to understand common sense reasoning and general knowledge of the world for performing reasoning tasks (e.g., common sense or logical reasoning); ¶¶ [0044]-[0046] with FIGS. 2 and 3A: the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; at operation 316, the language model generator 240 trains the small language model with the filtered general dataset; by curating the training data for the small language model, the small language model is trained with data that satisfies certain quality suitable for certain application; once the first training phase is performed, the small language model has general understanding of natural language; once the first training phase is performed, the small language model has general understanding of natural language; ¶ [0052] with FIGS. 2 and 3B: at operation 320, the language model generator 240 trains the small language model with the synthetic dataset; once the small language model went through the two-phase training, the small language model is adapted to understand common sense reasoning and general knowledge of the world for performing reasoning tasks (e.g., common sense or logical reasoning)).
BUBECK further discloses a machine-storage medium (BUBECK, ¶ [0027] with 224 in FIG.2: memory 224; ¶ [0038] with FIG. 3: a computer readable medium; ¶¶ [0069]-[0070] and [0075] with 504 and 510 in FIG. 5: a system memory 504; the non-removable storage device 510; computer readable media as used herein may include computer storage media; ¶ [0081] with 662 and 668 in FIG. 6: a non-volatile storage area 668 within the memory 662; ¶¶ [0104] and [0121]: a computer storage medium) for storing instructions that, when executed by one or more hardware processors (BUBECK, ¶ [0027] with 220 in FIG. 2: a processor 222; ¶ [0038] with FIG. 3: the method 300 can be performed by gates or circuits associated with a processor, Application Specific Integrated Circuit (ASIC), a field programmable gate array FPGA), a system on chip (SOC), a Neural Processing Unit (NPU), or other hardware device; ¶¶ [0069] and [0073] with 502 in FIG. 5: at least one processing unit 502; ¶¶ [0084] and [0102] with 660/661 in FIG. 6: the processor 660/specifical purpose processor 661), cause the one or more hardware processors to perform operations described above (BUBECK, ¶¶ [0037]-[0038] with FIGS. 2-3: the method 300 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium; e.g., the set of computer-executable instructions stored on a storage device may cause a computing device ( e.g., a server 230) to perform the method 300; the method 300 can be performed by gates or circuits associated with a processor, Application Specific Integrated Circuit (ASIC), a field programmable gate array (FPGA), a system on chip (SOC), a Neural Processing Unit (NPU), or other hardware device; ¶ [0072] with FIG. 5: a number of program modules and data files may be stored in the system memory 504; while executing on the processing unit 502, the program modules 506 ( e.g., application 520) may perform processes including, but not limited to, the aspects, as described herein; ¶ [0081] with FIG. 6: one or more application programs 666 may be loaded into the memory 662 and run on or in association with the operating system 664; ¶ [0104]: methods may be partially implemented in software that can be stored on a storage medium, executed on programmed general purpose computer with the cooperation of a controller and memory, a special purpose computer, a microprocessor, or the like).
BUBECK fails to explicitly disclose (1) identifying a plurality of data items associated with user-generated content; and (2) each label describing a sentiment of user-generated content associated with a respective data item.
Karamouzas teaches a system and a method relating to distilling knowledge (Karamouzas, ABSTRACT in Page 1), wherein identifying a plurality of data items associated with user-generated content; and each label describing a sentiment of user-generated content associated with a respective data item (Karamouzas, ABSTRACT in Page 1: sentiment analysis in texts, also known as opinion mining, is a significant Natural Language Processing (NLP) task, with many applications in automated social media monitoring, customer feedback processing, e-mail scanning, etc.; texts containing figurative language (e.g., sarcasm, irony, metaphors) still pose a challenge to existing methods due to the semantic ambiguities they entail; a novel setup of neural knowledge transfer is proposed for DNN-based sentiment analysis of figurative texts; it is employed for distilling knowledge from a pretrained binary recognizer of figurative language into a multiclass sentiment classifier, while the latter is being trained under a multitask setting; thus, hints about figurativeness implicitly help resolve semantic ambiguities; Section 1 in Pages 1-2: sentiment analysis, also known as opinion mining, is one of the most important tasks in the field of Natural Language Processing (NLP), due to its many different uses; in the simplest case, it is the problem of assigning a class label to a corpus of written text, that may range from a single phrase up to a complete essay, where each class expresses a possible sentiment of the author concerning the content of the text; thus, in typical scenarios, the class labels are “positive”, “neutral”, “negative”, etc.; opinion mining can be employed for tasks such as automated social media monitoring, customer feedback processing or e-mail scanning; the huge rise in the volume of subjectively written texts that are publicly available on-line during the past decade, such as user reviews or social media posts, has further increased the practical significance of accurate sentiment analysis methods; texts containing figurative language (e.g., involving sarcasm, irony, metaphors, etc.) still pose a challenge to existing methods; this is due to difficulties in identifying whether a figurative phrase actually implies negative or positive opinion, given its inherent semantic ambiguity; the issue is especially pronounced because of the ubiquitous nature of figurative language (FL) in human communication; research specifically focused on sentiment analysis for FL-heavy texts; a novel setup of knowledge distillation is proposed for DNN-based sentiment analysis of figurative texts; it is employed for transferring knowledge from a binary pretrained FL recognizer (teacher) into a multiclass sentiment classifier (student), while the latter is being trained under a multitask setting; up to now, knowledge distillation has mainly been applied in softmax-based classification settings, where the teacher and the student solve identical tasks; this paper, instead, proposes distillation from a binary classification teacher with sigmoidal output, in order to aid a multiclass classification student on a completely different task; the main contributions of this paper are: (a) an innovative way to apply neural distillation in NLP; and (b) exploiting neural distillation for increasing sentiment analysis accuracy in texts with FL; Section 2 in Page 2: Sentiment analysis on FL was originally proposed at the Semantic Evaluation Workshop 2015 - Task 11; given a set of tweets that are rich in metaphor, sarcasm and irony, the goal was to determine whether a user has expressed a positive, negative or neutral sentiment in each one of them; Knowledge distillation is the most prevalent form of neural knowledge transfer; it implies training a student DNN according to the response of a pretrained teacher DNN to input samples (so-called “transfer dataset”), possibly consisting of unknown/unlabeled data; typically, the student is assumed to be initially untrained (encoding no prior knowledge), while the two networks may have different neural architectures; the student DNN is trained with an objective function that penalizes the deviation of its output from the corresponding output of the fixed teacher DNN on the transfer set; the goal is to transfer so-called dark knowledge from the teacher to the student; hints from hidden layer activations in the teacher when processing the transfer set may also be employed for guiding student training [13], besides the final output; dimensionality reduction may be needed when the layer dimensions differ between student and teacher; to bypass this limitation, student training in [14] is guided by the similarities between transfer set data samples representations constructed by the teacher; intermediate “teacher assistant” networks, in the context of a multi-step process, was also proposed in [15] for scenarios where the two models differ in complexity; finally, distillation from a deep linear teacher for binary classification was presented in the context of a purely theoretical analysis in [16], while a similar investigation was conducted in [17] for shallow feed-forward neural networks; Section 3 with FIG. 1 in Pages 2-4: the proposed method exploits knowledge distillation for increasing DNN-based sentiment analysis accuracy on texts with FL, that employ sarcasm, irony and/or metaphor; thus, during training, a teacher-student architecture is utilized to enrich the student model with the knowledge of a pretrained FL recognizer; the latter one is a binary classifier, while the former one is a multiclass classifier tasked with identifying sentiment in input texts; thus, due to the nature of the teacher and the different task it solves compared to the student, an atypical kind of distillation is proposed that has not been previously employed in regular DNNs for real-world tasks; let us also assume that a DNN-based binary text classifier F has been pretrained under a regular supervised setting on a database containing two classes: “figurative”, “literal/non-figurative”; the student S is the neural model we actually want to optimize; on a different, sentiment-annotated dataset; without loss of generality, we assume that it is being trained under a supervised multiclass text classification setting; typically, N ≥ 3 classes are employed for the sentiment analysis/opinion mining task (“positive”, “neutral”, “negative”, etc.) and a final softmax activation layer used for deriving the class prediction; S is trained by a regular, suitable loss function LS, such as Cross-Entropy (CE); the proposed method consists in training S with the following multitask loss function:
L
M
=
L
S
+
α
L
D
, where
L
D
is being computed at each iteration by exploiting the pretrained F; α is a scalar hyperparameter balancing the two individual training objectives
L
S
and
L
D
;
L
D
essentially distills F(x): the output of F for input x, where x is the current training data point; as noted in [16] for the deep linear scenario, sigmoidal output activation for the binary classification case is equivalent to soft labels typically employed for softmax-based multiclass distillation; to compute this loss term, a parallel output layer sb serving as an auxiliary binary classification head is architecturally plugged onto the penultimate layer of S, while F(x) serves as real-valued/continuous substitute “ground-truth” for
L
D
; by employing the Mean Squared Error cost (MSE) for LD, Eq. (1) is concretized into:
L
M
=
L
S
s
m
x
,
x
~
+
α
s
b
x
-
F
(
x
)
2
, where
x
~
is the actual, sentiment ground-truth class label corresponding to x, in the context of multiclass classification; as previously mentioned,
L
S
can be any supervised loss function for classification, such as Cross-Entropy (CE); notably, no figurativeness ground-truth label is exploited or required to exist for x; an overview of the proposed method is depicted in Figure 1; importantly, no actual/real ground-truth annotation concerning the presence or type of FL is required or exploited while training S for sentiment analysis; of course, after S has been fully trained, both the entire F model and the auxiliary output layer/binary classification head sb can be safely discarded).
BUBECK and Karamouzas are analogous art because they are from the same field of endeavor, a system and a method relating to distilling knowledge. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Karamouzas to BUBECK. Motivation for doing so would expand the capability and application of knowledge transfer in different area.
Claims 2 and 12
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses wherein the first ML model comprises a large-scale Large Language Model having weights of more than 100 billion parameters (BUBECK, ¶¶ [0018]-[0021] with FIG. 1: Large Language Models (LLMs) have transformed the field of Natural Language Processing; LLMs uses a large scale of parameters and tokens for training data in achieving high levels of capability; the improvement from one generation of LLMs to the next seems at the moment to primarily stem from scale, with the most powerful models; nearing trillions of parameters and trillion of tokens for training data; however, the cost of training, deploying, and maintaining such large models may be substantial; from a responsible artificial intelligence (AI) standpoint, the energy consumption of large-scale models is becoming an increasing concern, as is the question of how controllable or governable these large models can be; a small language model (SLM) is generated via a two-phase training to achieve similar capabilities as large language models (LLMs); the small language model is a type of generative machine leaning model that are trained with smaller training data with fewer parameters compared to LLMs; specifically, the two-phase training includes a first training phase for teaching the small language model general knowledge and language understanding and a second training phase for teaching the small language model logical reasoning and various niche skills; the first training phase involves generating a filtered general dataset from various sources (e.g., web sources) and training the small language model with the filtered general dataset; the second training phase involves generating a synthetic dataset using prompts and training the small language model with the synthetic dataset; by curating the training data through the two-phase training from two separate sources, the small language model is trained with data that satisfy certain quality suitable for understanding common sense reasoning and general knowledge of the world and performing certain application and reasoning tasks (e.g., common sense or logical reasoning); prior to two-phase training, the small language model training may warm start 110 by combining trained weights with new initialized weights; e.g., the trained weights from an existing trained model may be copied into the new small language model; a tiling approach may be used to copy the weights between different dimensions of models to ensure that the weights follow the magnitude distribution of the previous small language model and that the structure of the weight matrices is enforced; by doing so, the warm start 110 may jumpstart on the training of a new model and have the knowledge transfer between the existing and new models to achieve a higher level of performance with less amount of training needed for the new model; this saves computational resources and improves generalizability when training the small language model).
Claims 4 and 14
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses wherein the third ML model comprises a small-scale Large Language Model having weights of less than 1 billion parameters (BUBECK, ¶¶ [0018]-[0021] with FIG. 1: Large Language Models (LLMs) have transformed the field of Natural Language Processing; LLMs uses a large scale of parameters and tokens for training data in achieving high levels of capability; the improvement from one generation of LLMs to the next seems at the moment to primarily stem from scale, with the most powerful models; nearing trillions of parameters and trillion of tokens for training data; however, the cost of training, deploying, and maintaining such large models may be substantial; from a responsible artificial intelligence (AI) standpoint, the energy consumption of large-scale models is becoming an increasing concern, as is the question of how controllable or governable these large models can be; a small language model (SLM) is generated via a two-phase training to achieve similar capabilities as large language models (LLMs); the small language model is a type of generative machine leaning model that are trained with smaller training data with fewer parameters compared to LLMs; specifically, the two-phase training includes a first training phase for teaching the small language model general knowledge and language understanding and a second training phase for teaching the small language model logical reasoning and various niche skills; the first training phase involves generating a filtered general dataset from various sources ( e.g., web sources) and training the small language model with the filtered general dataset; the second training phase involves generating a synthetic dataset using prompts and training the small language model with the synthetic dataset; by curating the training data through the two-phase training from two separate sources, the small language model is trained with data that satisfy certain quality suitable for understanding common sense reasoning and general knowledge of the world and performing certain application and reasoning tasks (e.g., common sense or logical reasoning); prior to two-phase training, the small language model training may warm start 110 by combining trained weights with new initialized weights; e.g., the trained weights from an existing trained model may be copied into the new small language model; a tiling approach may be used to copy the weights between different dimensions of models to ensure that the weights follow the magnitude distribution of the previous small language model and that the structure of the weight matrices is enforced; by doing so, the warm start 110 may jumpstart on the training of a new model and have the knowledge transfer between the existing and new models to achieve a higher level of performance with less amount of training needed for the new model; this saves computational resources and improves generalizability when training the small language model).
Claims 5 and 15
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses wherein the plurality of data items associated with user-generated content comprises one or more of a plurality of comments and a plurality of reviews (Karamouzas, ABSTRACT in Page 1: sentiment analysis in texts, also known as opinion mining, is a significant Natural Language Processing (NLP) task, with many applications in automated social media monitoring, customer feedback processing, e-mail scanning, etc.; texts containing figurative language (e.g., sarcasm, irony, metaphors) still pose a challenge to existing methods due to the semantic ambiguities they entail; a novel setup of neural knowledge transfer is proposed for DNN-based sentiment analysis of figurative texts; it is employed for distilling knowledge from a pretrained binary recognizer of figurative language into a multiclass sentiment classifier, while the latter is being trained under a multitask setting; thus, hints about figurativeness implicitly help resolve semantic ambiguities; Section 1 in Pages 1-2: sentiment analysis, also known as opinion mining, is one of the most important tasks in the field of Natural Language Processing (NLP), due to its many different uses; in the simplest case, it is the problem of assigning a class label to a corpus of written text, that may range from a single phrase up to a complete essay, where each class expresses a possible sentiment of the author concerning the content of the text; thus, in typical scenarios, the class labels are “positive”, “neutral”, “negative”, etc.; opinion mining can be employed for tasks such as automated social media monitoring, customer feedback processing or e-mail scanning; the huge rise in the volume of subjectively written texts that are publicly available on-line during the past decade, such as user reviews or social media posts, has further increased the practical significance of accurate sentiment analysis methods; texts containing figurative language (e.g., involving sarcasm, irony, metaphors, etc.) still pose a challenge to existing methods; this is due to difficulties in identifying whether a figurative phrase actually implies negative or positive opinion, given its inherent semantic ambiguity; the issue is especially pronounced because of the ubiquitous nature of figurative language (FL) in human communication; research specifically focused on sentiment analysis for FL-heavy texts; Section 2.1 in Page 2: sentiment analysis on FL was originally proposed at the Semantic Evaluation Workshop 2015 - Task 11; given a set of tweets that are rich in metaphor, sarcasm and irony, the goal was to determine whether a user has expressed a positive, negative or neutral sentiment in each one of them; Section 3 with in Pages 2-4: the student S is the neural model we actually want to optimize; on a different, sentiment-annotated dataset; typically, N ≥ 3 classes are employed for the sentiment analysis/opinion mining task (“positive”, “neutral”, “negative”, etc.) and a final softmax activation layer used for deriving the class prediction).
Claims 6 and 16
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses wherein the sentiment of user-generated content corresponds to a model output value representing positive, negative, or neutral (Karamouzas, ABSTRACT in Page 1: sentiment analysis in texts, also known as opinion mining, is a significant Natural Language Processing (NLP) task, with many applications in automated social media monitoring, customer feedback processing, e-mail scanning, etc.; texts containing figurative language (e.g., sarcasm, irony, metaphors) still pose a challenge to existing methods due to the semantic ambiguities they entail; a novel setup of neural knowledge transfer is proposed for DNN-based sentiment analysis of figurative texts; it is employed for distilling knowledge from a pretrained binary recognizer of figurative language into a multiclass sentiment classifier, while the latter is being trained under a multitask setting; thus, hints about figurativeness implicitly help resolve semantic ambiguities; Section 1 in Pages 1-2: sentiment analysis, also known as opinion mining, is one of the most important tasks in the field of Natural Language Processing (NLP), due to its many different uses; in the simplest case, it is the problem of assigning a class label to a corpus of written text, that may range from a single phrase up to a complete essay, where each class expresses a possible sentiment of the author concerning the content of the text; thus, in typical scenarios, the class labels are “positive”, “neutral”, “negative”, etc.; opinion mining can be employed for tasks such as automated social media monitoring, customer feedback processing or e-mail scanning; the huge rise in the volume of subjectively written texts that are publicly available on-line during the past decade, such as user reviews or social media posts, has further increased the practical significance of accurate sentiment analysis methods; texts containing figurative language (e.g., involving sarcasm, irony, metaphors, etc.) still pose a challenge to existing methods; this is due to difficulties in identifying whether a figurative phrase actually implies negative or positive opinion, given its inherent semantic ambiguity; the issue is especially pronounced because of the ubiquitous nature of figurative language (FL) in human communication; research specifically focused on sentiment analysis for FL-heavy texts; Section 2.1 in Page 2: sentiment analysis on FL was originally proposed at the Semantic Evaluation Workshop 2015 - Task 11; given a set of tweets that are rich in metaphor, sarcasm and irony, the goal was to determine whether a user has expressed a positive, negative or neutral sentiment in each one of them; Section 3 with in Pages 2-4: the student S is the neural model we actually want to optimize; on a different, sentiment-annotated dataset; typically, N ≥ 3 classes are employed for the sentiment analysis/opinion mining task (“positive”, “neutral”, “negative”, etc.) and a final softmax activation layer used for deriving the class prediction; Section 4.2 in Pages 4-5: the S15-T11 dataset [3] was used for evaluating the proposed method and comparing it against competing approaches; it contains 8000/4000 tweets for training/test, respectively, including tweets with ironic, sarcastic and metaphorical language; the 12000 data points are grouped under 11 classes annotated with integers in an 11-point scale, ranging from -5 to +5, that denote the polarity of each tweet, from “very negative” to “very positive”; since it is a sentiment analysis dataset, it does not contain ground-truth annotations/labels concerning the presence or type of FL).
Claims 7 and 17
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses determining a confidence value based on the first plurality of labels for the first subset of data items; training the second ML model and the third ML model based on the confidence value (BUBECK, ¶¶ [0054]-[0067] with FIGS. 4A-4B: conceptual diagram 400 in FIG. 4A depicts an overview of pre-trained generative model package 404 that processes an input 402 to generate model output for capturing and generatively transforming content items from a generative model output 406; generative model package 404 may be more generally pre-trained, such that input 402 includes a prompt that is generated, selected, or otherwise engineered to induce generative model package 404 to produce certain generative model output 406; generative model package 404 includes input tokenization 408, input embedding 410, model layers 412, output layer 414, and output decoding 416; FIG. 4B is a conceptual diagram that depicts an example architecture 450 of a pre-trained generative machine learning model that may be used; architecture 450 processes input 402 to produce generative model output 406; architecture 450 is depicted as a transformer model that includes encoder 452 and decoder 454; Softmax 494 may further convert the output of linear transformation 492 to predicted next-token probabilities, as indicated by output probabilities 496; output probabilities 496 may thus form model output 406; it is also well known in the art that probability is used to indicate the confidence value1; ¶¶ [0004]-[0007] and [0019]-[0020]: by curating the training data through the two-phase training from two separate sources, the small language model is trained with data that satisfy certain quality suitable for understanding common sense reasoning and general knowledge of the world and performing certain application and reasoning tasks (e.g., common sense or logical reasoning); annotating a subset of the general dataset based on one or more classifier metrics indicative of a quality of the general dataset; training a classifier based on the annotated subset of the general dataset and the one or more classifier metrics; analyzing each general data of the general dataset to determine a score for each of the one or more classifier metrics associated with the respective general data using the trained classifier, generating a filtered general dataset by filtering the general dataset based on one or more filters, the one or more filters indicative of threshold scores for corresponding classifier metrics, training the small language model with the filtered general dataset; ¶¶ [0022]-[0024] with FIG. 1: once the general dataset is obtained, a subset of the general dataset that represents the general dataset is selected and an annotated version of the subset of the general dataset is generated based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; each general data of the subset of the general dataset is annotated with a score for each classifier metric of the one or more classifier metrics using the generative transformer; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; once the classifier is trained, each general data of the general dataset is analyzed to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, one or more filters for filtering the general dataset are generated; each filter indicates a threshold score for a respective classifier metric; in other words, the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the filtered general dataset is generated by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model. The filtered general dataset is used to train the small language model in the first training phase 120; ¶¶ [0029]-[0032] with FIG. 2: the general dataset manager 242 is configured to select a subset of the general dataset that represents the general dataset and generate an annotated version of the subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; the general dataset manager 242 is configured to annotate each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; upon annotating the subset of the general dataset, the general dataset manager 242 is configured to train a classifier using the annotated subset of the general dataset; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; once the classifier is trained, the general dataset manager 242 is further configured to analyze each general data of the general dataset; specifically, the general dataset manager 242 is configured to analyze each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the general dataset manager 242 is configured to generate one or more filters for filtering the general dataset; each filter indicates a threshold score for a respective classifier metric; i.e., the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the general dataset manager 242 is configured to generate the filtered general dataset; specifically, the general dataset manager 242 is configured to generate the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; ¶¶ [0041]-[0046] with FIGS. 2 and 3A: at operation 306, the language model generator 240 annotates a subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a subset of the general dataset is selected to represent the general dataset; once the subset of the general dataset is selected, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; the language model generator 240 annotates each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; upon annotating the subset of the general dataset, the language model generator 240 trains a classifier using the annotated subset of the general dataset as indicated in operation 308; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; subsequent to training the classifier, the language model generator 240 analyzes each general data of the general dataset as indicated in operation 310; specifically, the language model generator 240 analyzes each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the language model generator 240 generates one or more filters for filtering the general dataset as indicated in operation 312; each filter indicates a threshold score for a respective classifier metric; i.e., the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the language model generator 240 generates a filtered general dataset as indicated in operation 314; specifically, the language model generator 240 generates the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; at operation 316, the language model generator 240 trains the small language model with the filtered general dataset; by curating the training data for the small language model, the small language model is trained with data that satisfies certain quality suitable for certain application).
Claims 9 and 19
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 7 and 17 respectively and further discloses configuring a model output probability based on the confidence value; and training the second ML model and the third ML model based on the model output probability (BUBECK, ¶¶ [0054]-[0067] with FIGS. 4A-4B: conceptual diagram 400 in FIG. 4A depicts an overview of pre-trained generative model package 404 that processes an input 402 to generate model output for capturing and generatively transforming content items from a generative model output 406; generative model package 404 may be more generally pre-trained, such that input 402 includes a prompt that is generated, selected, or otherwise engineered to induce generative model package 404 to produce certain generative model output 406; generative model package 404 includes input tokenization 408, input embedding 410, model layers 412, output layer 414, and output decoding 416; FIG. 4B is a conceptual diagram that depicts an example architecture 450 of a pre-trained generative machine learning model that may be used; architecture 450 processes input 402 to produce generative model output 406; architecture 450 is depicted as a transformer model that includes encoder 452 and decoder 454; Softmax 494 may further convert the output of linear transformation 492 to predicted next-token probabilities, as indicated by output probabilities 496; output probabilities 496 may thus form model output 406; it is also well known in the art that probability is used to indicate the confidence value2; ¶¶ [0004]-[0007] and [0019]-[0020]: by curating the training data through the two-phase training from two separate sources, the small language model is trained with data that satisfy certain quality suitable for understanding common sense reasoning and general knowledge of the world and performing certain application and reasoning tasks (e.g., common sense or logical reasoning); annotating a subset of the general dataset based on one or more classifier metrics indicative of a quality of the general dataset; training a classifier based on the annotated subset of the general dataset and the one or more classifier metrics; analyzing each general data of the general dataset to determine a score for each of the one or more classifier metrics associated with the respective general data using the trained classifier, generating a filtered general dataset by filtering the general dataset based on one or more filters, the one or more filters indicative of threshold scores for corresponding classifier metrics, training the small language model with the filtered general dataset; ¶¶ [0022]-[0024] with FIG. 1: once the general dataset is obtained, a subset of the general dataset that represents the general dataset is selected and an annotated version of the subset of the general dataset is generated based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; each general data of the subset of the general dataset is annotated with a score for each classifier metric of the one or more classifier metrics using the generative transformer; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; once the classifier is trained, each general data of the general dataset is analyzed to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, one or more filters for filtering the general dataset are generated; each filter indicates a threshold score for a respective classifier metric; in other words, the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the filtered general dataset is generated by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model. The filtered general dataset is used to train the small language model in the first training phase 120; ¶¶ [0029]-[0032] with FIG. 2: the general dataset manager 242 is configured to select a subset of the general dataset that represents the general dataset and generate an annotated version of the subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; the general dataset manager 242 is configured to annotate each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; upon annotating the subset of the general dataset, the general dataset manager 242 is configured to train a classifier using the annotated subset of the general dataset; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; once the classifier is trained, the general dataset manager 242 is further configured to analyze each general data of the general dataset; specifically, the general dataset manager 242 is configured to analyze each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the general dataset manager 242 is configured to generate one or more filters for filtering the general dataset; each filter indicates a threshold score for a respective classifier metric; i.e., the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the general dataset manager 242 is configured to generate the filtered general dataset; specifically, the general dataset manager 242 is configured to generate the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; ¶¶ [0041]-[0046] with FIGS. 2 and 3A: at operation 306, the language model generator 240 annotates a subset of the general dataset based on one or more classifier metrics indicative of a quality of the subset of the general dataset; to do so, a subset of the general dataset is selected to represent the general dataset; once the subset of the general dataset is selected, a generative transformer is used for annotations on the quality of the subset of the general dataset based on one or more classifier metrics; the language model generator 240 annotates each general data of the subset of the general dataset with a score for each classifier metric of the one or more classifier metrics using the generative transformer; upon annotating the subset of the general dataset, the language model generator 240 trains a classifier using the annotated subset of the general dataset as indicated in operation 308; the trained classifier is configured to predict quality of data based on the one or more classifier metrics; e.g., the quality of data may be represented by a score for each classifier metric; subsequent to training the classifier, the language model generator 240 analyzes each general data of the general dataset as indicated in operation 310; specifically, the language model generator 240 analyzes each general data to determine a score for each classifier metric associated with the respective general data using the trained classifier; based on the scores of the general data, the language model generator 240 generates one or more filters for filtering the general dataset as indicated in operation 312; each filter indicates a threshold score for a respective classifier metric; i.e., the one or more filters may be used to filter the general dataset to select general data from the general dataset that have with certain attributes or features for training the small language model; once the one or more filters are generated, the language model generator 240 generates a filtered general dataset as indicated in operation 314; specifically, the language model generator 240 generates the filtered general dataset by filtering the general dataset using the one or more filters; the filtered general dataset is a subset of the general dataset that satisfies a predefined level of quality; in aspects, one or more filters are selected to obtain those general data that have certain attributes or features for training the small language model; at operation 316, the language model generator 240 trains the small language model with the filtered general dataset; by curating the training data for the small language model, the small language model is trained with data that satisfies certain quality suitable for certain application).
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over BUBECK in view of Karamouzas as applied to Claims 1 and 11 respectively above, and further in view of Mirzadeh et al. ("Improved Knowledge Distillation via Teacher Assistant",arXiv:1902.03393v2, dec 17, 2019, pp. 1-11), hereinafter Mirzadeh and Hu et al. ("Large Language Model Meets Graph Neural Network in Knowledge Distillation", arXiv:2402.05894v2, Feb. 9, 2024, pp. 1-17), hereinafter Hu'2024.
Claims 3 and 13
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 1 and 11 respectively and further discloses in ¶¶ [0018]-[0021] with FIG. 1 that Large Language Models (LLMs) have transformed the field of Natural Language Processing; LLMs uses a large scale of parameters and tokens for training data in achieving high levels of capability; the improvement from one generation of LLMs to the next seems at the moment to primarily stem from scale, with the most powerful models; nearing trillions of parameters and trillion of tokens for training data; however, the cost of training, deploying, and maintaining such large models may be substantial; from a responsible artificial intelligence (AI) standpoint, the energy consumption of large-scale models is becoming an increasing concern, as is the question of how controllable or governable these large models can be; a small language model (SLM) is generated via a two-phase training to achieve similar capabilities as large language models (LLMs); the small language model is a type of generative machine leaning model that are trained with smaller training data with fewer parameters compared to LLMs; specifically, the two-phase training includes a first training phase for teaching the small language model general knowledge and language understanding and a second training phase for teaching the small language model logical reasoning and various niche skills; the first training phase involves generating a filtered general dataset from various sources (e.g., web sources) and training the small language model with the filtered general dataset; the second training phase involves generating a synthetic dataset using prompts and training the small language model with the synthetic dataset; by curating the training data through the two-phase training from two separate sources, the small language model is trained with data that satisfy certain quality suitable for understanding common sense reasoning and general knowledge of the world and performing certain application and reasoning tasks (e.g., common sense or logical reasoning); prior to two-phase training, the small language model training may warm start 110 by combining trained weights with new initialized weights; e.g., the trained weights from an existing trained model may be copied into the new small language model; a tiling approach may be used to copy the weights between different dimensions of models to ensure that the weights follow the magnitude distribution of the previous small language model and that the structure of the weight matrices is enforced; and by doing so, the warm start 110 may jumpstart on the training of a new model and have the knowledge transfer between the existing and new models to achieve a higher level of performance with less amount of training needed for the new model; this saves computational resources and improves generalizability when training the small language model.
BUBECK in view of Karamouzas fails to explicitly disclose wherein the second ML model comprises a medium-scale Large Language Model having weights between 1 billion parameters and 100 billion parameters.
Mirzadeh teaches a system and a method relating to Knowledge Distillation (Mirzadeh, Title and Abstract in Page 1), wherein the second ML model comprises a medium-scale (Mirzadeh, Abstract in Page 1: despite the fact that deep neural networks are powerful models and achieve appealing results on many tasks, they are too large to be deployed on edge devices like smartphones or embedded sensor nodes; there have been efforts to compress these networks, and a popular method is knowledge distillation, where a large (teacher) pre-trained network is used to train a smaller (student) network; however, in this paper, we show that the student network performance degrades when the gap between student and teacher is large; given a fixed student network, one cannot employ an arbitrarily large teacher, or in other words, a teacher can effectively transfer its knowledge to students up to a certain size, not smaller; to alleviate this shortcoming, we introduce multi-step knowledge distillation, which employs an intermediate-sized network (teacher assistant) to bridge the gap between the student and the teacher; moreover, we study the effect of teacher assistant size and extend the framework to multi-step distillation; Introduction in Page 1: propose a new distillation framework called Teacher Assistant Knowledge Distillation (TAKD), which introduces intermediate models as teacher assistants (TAs) between the teacher and the student to fill in their gap (Figure 1); TA models are distilled from the teacher, and the student is then only distilled from the Tas; propose a teacher assistant based knowledge distillation approach to improve the accuracy of student network in the case of extreme compression; extend this framework to include a chain of multiple Tas from teacher to student to further improve the knowledge transfer and provided some insights to find the best one; extensive empirical evaluations and a theoretical justification, we show that introducing intermediary TA networks improves the distillation performance; Assistant based Knowledge Distillation with FIGS. 2-3 in Pages 2-4: with increasing teacher size, its own (test) accuracy increases (plotted in red on the right axis); however, the trained student accuracy first increases and then decreases (depicted in blue on the left axis); to explain this phenomenon, we can name a few factors that are competing against each other when enlarging the teacher: a. teacher’s performance increases, thus it provides better supervision for the student by being a better predictor; b. the teacher is becoming so complex that the student does not have the sufficient capacity or mechanics to mimic her behavior despite receiving hints; and c. teacher’s certainty about data increases, thus making its logits (soft targets) less soft, and this weakens the knowledge transfer which is done via matching the soft targets; in this paper, we propose to use intermediate-size networks to fill in the gap between them; the teacher assistant (TA) lies somewhere in between teacher and student in terms of size or capacity; first, the TA network is distilled from the teacher; then, the TA plays the role of a teacher and trains the student via distillation; this strategy will alleviate factor 2 in the previous subsection by being closer to the student than the teacher; therefore, the student is able to fit TA’s logit distribution more effectively than that of the teacher’s; it also alleviates factor 3 by allowing softer (and maybe) less confident targets; in terms of factor 1, a TA may degrade the performance, however, as we will see in experiments and theoretical analysis sections, both empirical results and theoretical analyses substantiate the effectiveness (improved performance) of TAKD; this happens because encouraging positively correlated factors (like 2 and 3) outweighs the performance loss due to negative ones (like 1)).
BUBECK in view of Karamouzas, and Mirzadeh are analogous art because they are from the same field of endeavor, a system and a method relating to Knowledge Distillation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Mirzadeh to BUBECK in view of Karamouzas. Motivation for doing so would improve the accuracy of student network in the case of extreme compression (Mirzadeh, Introduction in Page 1).
BUBECK in view of Karamouzas and Mirzadeh fails to explicitly disclose a medium-scale Large Language Model having weights between 1 billion parameters and 100 billion parameters.
Hu'2024 teaches a system and a method relating to Knowledge Distillation (Hu'2024, TITLE and ABSTRACT in Page 1), a medium-scale Large Language Model having weights between 1 billion parameters and 100 billion parameters (Hu'2024, ABSTRACT in Page 1: propose a novel graph knowledge distillation framework, termed Linguistic Graph Knowledge Distillation (LinguGKD), using LLMs as teacher models and GNNs as student models for knowledge distillation; it involves TAG-oriented instruction tuning of LLM on designed tailored prompts, followed by propagating knowledge and aligning the hierarchically learned node features from the teacher LLM to the student GNN in latent space, employing a layer-adaptive contrastive learning strategy; Section 1 in Pages 1-2: knowledge distillation stands out as a promising strategy, enabling the transfer of LLM insights to more compact GNNs; the idea not only aims to reduce model size and computational demands of LLM but also capitalizes on GNNs’ strengths in processing structural data; by integrating the semantic understanding of LLMs with the structural mining of GNNs, it is expected to optimize graph reasoning tasks, facilitating more effective and adaptable real-world applications; under the above motivation, we explore the innovative integration of graph knowledge distillation from complex LLMs as teachers to lightweight GNNs as students, with a focus on node classification tasks; acknowledging the inherent architectural differences and the distinct vector spaces of LLMs and GNNs, we introduce a novel framework-Linguistic Graph Knowledge Distillation (LinguGKD); it begins by instruction tuning a pre-trained LLM (PLM) with carefully designed node classification instruction prompts; it then employs a layer-adaptive contrastive distillation strategy, complemented by a feature alignment mechanism to synchronize LLM and GNN feature spaces, transferring hierarchical node features from LLM to GNN; by doing so, it can propagate the teacher LLM’s deep semantic knowledge and intricate understanding of graph structures to the student GNN, leading to better performance of node classification in TAG; Section 2 in Pages 2-3: while GNNs excel at processing graph structures, they have limitations in understanding individual node semantics; taking advantage of the capability of LLM’s semantic feature learning, LLM-based Graph Knowledge Distillation can be designed by distilling knowledge from a teacher LLM to a student GNN, enhancing the GNN’s capability in interpreting node semantics and complex graph topology; Section 3 with FIG. 1 in Pages 3-5: Figure 1 showcases the LinguGKD framework for TAG-oriented graph knowledge distillation, highlighting three key components: teacher feature learning by the LLM, student feature learning by the GNN, and layer-adaptive contrastive distillation loss between the two feature sets; we elaborate three pivotal phases of LinguGKD’s knowledge distillation process: (a) extracting semantically-enhanced node features via LLMs; (b) leveraging GNNs for structural node feature extraction; and (c) implementing layer-adaptive alignment of semantic and structural features; in training the student GNN, acknowledging the different inference frameworks of the teacher and student models, it’s essential to not only distill knowledge from the teacher LLM but also to train a task-specific prediction layer for the GNN. Thus, we approach the student GNN’s training as a multi-task joint optimization challenge; the overall training objective integrates the knowledge distillation loss L𝐷 with the classification loss L𝐺, obtaining a joint loss function; finally, the student GNN undergoes end-to-end training using a mini-batch AdamW optimizer, which is dedicated to optimizing model parameters efficiently for robust performance; Section 4.2 with FIG. 2 in Pages 6-8: as shown in Figure 2, the right y-axis delineates the parameter quantity and storage requirements of various models through a line graph; concerning the number of model parameters, we reveal that while LLMs like Llama-7B and Mistral-7B encompass approximately 7 billion parameters, traditional GNNs range from 7 million to 11 million parameters, depending on the design difference of their message-passing layers).
BUBECK in view of Karamouzas and Mirzadeh, and Hu'2024 are analogous art because they are from the same field of endeavor, a system and a method relating to Knowledge Distillation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Hu'2024 to BUBECK in view of Karamouzas and Mirzadeh. Motivation for doing so would facilitate more effective and adaptable real-world applications (Hu'2024, Section 1 in Pages 1-2).
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over BUBECK in view of Karamouzas as applied to Claims 7 and 17 respectively above, and further in view of Zhdanov et al. (US 11,048,979 B1, Date of Patent: 06/29/2021), hereinafter Zhdanov'979.
Claims 8 and 18
BUBECK in view of Karamouzas discloses all the elements as stated in Claims 7 and 17 respectively and except failing to explicitly disclose wherein the confidence value represents an accuracy of annotation for the first subset of data items.
Zhdanov'979 teaches a system and a method relating to data labeling (Zhdanov, ABSTRACT), wherein the confidence value represents an accuracy of annotation for the first subset of data items (Zhdanov'979, Col. 9, line 62 – Col. 10, line 52 with FIG. 4: active learning service 402 can use one or more active learning algorithms and one or more machine learning models 406 to determine a portion of the dataset to be auto-annotated and a portion of the dataset to be manually annotated; the machine learning model can output a confidence score for the feature it is trained to identify in each item of data in the dataset (e.g., each image in an image dataset, each word in a text dataset, etc.); a threshold can be set above which auto-annotation can be performed with an acceptable level of accuracy; in addition to the thresholding, a statistical analysis of the model can be performed to determine confidence bounds associated with the model; e.g., a validation dataset, which includes known features, can be passed through the model; because the features of the validation dataset are known, the accuracy of the model can be evaluated; the confidence scores output for each feature in the dataset can be arranged from the least to the most confidence; the accuracy of subsets of the dataset can then be evaluated; e.g., the accuracy of each decile of the outputs can be determined; this results in a chart that shows you the confidence scores grouped by decile (or other subset) on the X axis and the Y axis shows you the accuracy for that decile; a regression, such as Gaussian Process Regression, can be performed to obtain a smooth curve based on the accuracy of each subset; as a result, each confidence score has a corresponding accuracy value on the smooth curve and corresponding confidence bounds for that accuracy; the threshold can then be determined based on a user's choice of acceptable accuracy; features identified with a confidence score with a corresponding accuracy above the threshold can be auto-labeled 404 with the labels generated by the machine learning model 406; features identified with a confidence score below a threshold may be sent by active learning service 402 to be manually labeled 408; manual labeling 408 can include sending a subset of the unlabeled dataset to a workforce interface service which may interface with one or more types of manual annotators; these annotations may then be consolidated into labels by annotation consolidation service 410; these annotations may be consolidated based on, e.g., the accuracy of the annotator and other factors) (Zhdanov, Col. 3, line 55 – Col. 4, line 35 with FIG. 1: a gold standard dataset may refer to a dataset that has been verified as being accurately labeled; the customer may also specify one or more of a gold standard dataset, a target label space, a desired quality threshold, and an annotation budget; labels are the outputs of annotations after the annotations have been consolidated and have achieved a quality score above a given threshold; the customer can specify a workflow to be used for labeling; the workflow may be provided by the active learning-based labeling service or may be a custom workflow specified by the user; each workflow can be a nested workflow comprising a series of steps corresponding to one or more of active learning, machine annotating, manual annotating, annotation consolidation and quality score computation, model training, and then outputting of training datasets and/or models; Col. 5, line 60 – Col. 7, line 61 with FIG. 1: annotation consolidation may refer to the process of taking annotations from multiple annotators ( e.g., humans and/or machines) and consolidating these together (e.g., using majority-consensus heuristics, removing bias or low-quality annotators, using probabilistic distribution that minimizes a risk function for observed, predicted and true labels, or other techniques); the annotation consolidation service 122 can maintain a label score and a worker score when 10 performing annotation consolidation; during consolidation, annotation consolidation service 122 can fetch current label scores for each piece of data in the dataset ( e.g., image, video frame, audio utterance, etc.) and current worker scores for the annotator who provided the annotations on that piece of data; a new label score and worker scores can be computed for the piece of data based on previous state and currently received new annotation; the label score can be compared to a specified threshold; if the label score is higher than the threshold then no additional annotations are required; if the label score is lower than the threshold then the data may be passed to additional annotators to be further annotated by annotating service 120; the data labeling service 108 may also output various performance metrics, such as performance against the annotation budget, quality score of annotated labels and performance against the defined quality threshold, logs and metrics in a monitoring dashboard, and/or an audit trail of annotations tasks as performed by annotators; the quality score can be a numerically computed value to measure the quality of consolidated labels; this score is compared to a desired quality threshold (which may be a default value or may be provided by the customer as an input) to determine whether consolidated labels have achieved the desired quality level and hence are considered trained labels; this quality score can be expressed as an output of a risk function on a probability distribution for real, true and predicted labels; the main loop starts by running inference with the model on the validation dataset; after that, every object is given a confidence level; e.g., for image classification, a confidence level can be the probability that the model predicts for the most confident class; a threshold is found, such that for the objects whose model confidence is above the threshold, the expected quality of the label provided by the model exceeds a required quality value provided by the customer; then, the inference on the unlabeled data is performed, and the threshold is applied on the resulting inferences; all objects with the confidence larger than the threshold get auto-annotated and put into the labeled dataset; for the rest of the objects, active learning is performed to identify a subset of the objects which potentially benefit the model most; active learning may be executed in a separate active learning loop that iteratively labels portions of the unlabeled dataset and trains the model to better auto-label the dataset; this batch of objects is sent for human annotation; Col. 12, line 45 – Col. 13, line 25 with FIG. 6: generating a plurality of auto-annotations corresponding to the plurality of objects in the validation dataset using the machine learning model, determining a plurality of accuracy values for the plurality of auto-annotations for the validation dataset, performing regression on the accuracy of the plurality of auto-annotations to determine a confidence interval for each accuracy value, and determining the threshold value based on the confidence interval for a selected accuracy value; calculating a quality score associated with the plurality of labels, determining the quality score is less than a quality threshold, and requesting additional annotation of the subset of the input dataset from the annotation service; requesting additional annotation may include requesting a number of annotators to perform the additional annotation, the number of annotators determined based at least on a difference between the quality score and the quality threshold; he operations may include calculating a quality score for the plurality of labels, determining the quality score is below a quality threshold, and sending the subset of images to the annotation service to be manually annotated by a number of annotators determined based on a difference between the quality score and the quality threshold).
BUBECK in view of Karamouzas, and Zhdanov'979 are analogous art because they are from the same field of endeavor, a system and a method relating to data labeling. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Zhdanov'979 to BUBECK in view of Karamouzas. Motivation for doing so would enable a user to build and manage large, high accuracy datasets for use in various machine learning systems without a time consuming and expensive process (Zhdanov'979, ABSTRACT; Col. 2, lines 29-61).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over BUBECK in view of Karamouzas as applied to Claim 1 above, and further in view of Lin et al. ("Pretrained Transformers for Text Ranking: BERT and Beyond", arXiv:2010.06467v3, Aug. 19, 2021, pp. 1-204), hereinafter Lin.
Claim 10
BUBECK in view of Karamouzas discloses all the elements as stated in Claim 1 and except failing to explicitly disclose wherein the third ML model comprises Bidirectional Encoder Representations from Transformers (BERT).
Lin teaches a system and a method relating to knowledge distillation (Lin, Section 3 in Pages 46-47 and Section 3.5.1 in Pages 98-101), wherein the third ML model comprises Bidirectional Encoder Representations from Transformers (BERT) (Lin, Section 3 in Pages 46-51: finally, we describe a number of efforts that attempt to go beyond BERT, to build ranking models that are faster (i.e., achieve lower inference latency), are better (i.e., obtain higher ranking effectiveness), or realize an interesting tradeoff between effectiveness and efficiency (Section 3.5); we cover ranking models that exploit knowledge distillation to train more compact student models and other transformer architectures, including ground-up redesign efforts and adaptations of pretrained sequence-to-sequence models; Section 3.1 with Table 4 in Pages 47-51: while the original paper [Devlin et al., 2019] presented only the BERTBase and BERTLarge configurations, with 12 and 24 transformer encoder layers, respectively, in later work Turc et al. [2019] pretrained a greater variety of model sizes with the help of knowledge distillation; these are all shown in Table 4; in general, size correlates with effectiveness in downstream tasks, and thus these configurations are useful for exploring effectiveness/efficiency tradeoffs; Section 3.4.3 in Pages 94-97: multi-stage ranking pipelines exploit faster (and possibly less effective) models in earlier stages to discard likely non-relevant documents so there are fewer candidates under consideration by more expensive models in later stages; in Section 3.5, we cover knowledge distillation and other threads of research in this broad space. Here, we discuss a solution that shares similar motivations, but is clearly inspired by multi-stage ranking architectures; Section 3.5 in Pages 97-98: building on our previous discussion of simple BERT variants in Section 3.2.2, we begin by discussing efforts to distill BERT into smaller models in Section 3.5.1; distilled models are similar to the simple BERT variants in that they can easily be “swapped in” as a replacement for BERT "classic"; Section 3.5.1 with Table 24 in Pages 98-101: knowledge distillation refers to a general set of techniques where a smaller student model learns to mimic the behavior of a larger teacher model; the goal is for the student model to achieve comparable effectiveness on a particular task but more efficiently (e.g., lower inference latencies, fewer model parameters, etc.); these ideas have been extended by many others [Sun et al., 2019a, Liu et al., 2019b, Sanh et al., 2019, Hofstätter et al., 2020], with a range of different student models, including smaller versions of BERT; unsurprisingly, knowledge distillation has been applied to text ranking; researchers have investigated whether the efficiency of BERT can be improved by distilling a larger trained (BERT) model into a smaller (but still BERT-based) one [Gao et al., 2020c, Li et al., 2020a, Chen et al., 2021, Zhang et al., 2020f]; to encourage the student model to mimic the behavior of the teacher model, one common distillation objective is the mean squared error between the student’s and teacher’s logits [Tang et al., 2019, Tahami et al., 2020]; the student model can be fine-tuned with the linear combination of the student model’s cross-entropy loss and the distillation objective as the overall loss shown in Eqn. 39; as another approach, TinyBERT proposed a distillation objective that additionally considers the mean squared error between the two models’ embedding layers, transformer hidden states, and transformer attention matrices [Jiao et al., 2019]; in the context of text ranking, Chen et al. [2021] reported that this more complicated objective can improve effectiveness; Gao et al. [2020c] observed that distillation can be applied to both a BERT model that has already been fine-tuned for relevance classification (“ranker distillation”) and to pretrained but not yet fine-tuned BERT itself (“LM distillation”); concretely, this yields three possibilities: (a) apply distillation so that a (randomly initialized) student model learns to directly mimic an already fine-tuned teacher model using the distillation objective above (“ranker distillation”), (b) apply LM distillation into a student model followed by fine-tuning the student model for the relevance classification task (“LM distillation + fine-tuning”), or (c) apply LM distillation followed by ranker distillation (“LM + ranker distillation”); the “Distill” versions of these models perform comparably to the original BERTBase version, indicating that distillation into a “slightly smaller” model can improve efficiency without harming effectiveness)
BUBECK in view of Karamouzas, and Lin are analogous art because they are from the same field of endeavor, a system and a method relating to knowledge distillation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Lin to BUBECK in view of Karamouzas. Motivation for doing so would improve efficiency without harming effectiveness (Lin, Section 3.5.1 with Table 24 in Pages 98-101).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
1 See, for example US Patent No. 11,048, 979 B1 to Zhdanov et al., Date of Patent: 06/29/2021, Col. 7, lines 37-39 that "for image classification, a confidence level can be the probability that the model predicts for the most confident class".
2 See, for example US Patent No. 11,481,906 B1 to Narayanan et al., Date of Patent: 10/25/2022, Col. 7, lines 35-37 that "for image classification, a confidence level can be the probability that the model predicts for the most confident class".
