DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Examiner notes that the application is a continuation of US Patent No. 12008459, filed 6/17/2019 which, claims the benefit of prior-filed a U.S. Provisional Application No. 62/836,542, filed on 04/19/2019.
Drawings
The drawings were received on 05/03/2024. These drawings are acceptable.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 40 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 26 of U.S. Patent No. 12008459. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims in the patent anticipate the boarder limitation in the instant application. See further details in the table below.
US Patent No. 12008459, hereinafter ‘RefDoc’ teaches the limitations as highlighted in the table and analysis below:
U.S. Application No. 18136780
Examiner notes:
U.S. Patent No. 12008459
(Reference Patent, hereinafter ‘RefDoc’)
Claim 40
A computer-readable storage medium storing computer-readable instructions which, when executed by a hardware processing unit, cause the hardware processing unit to perform acts comprising:
providing a multi-task natural language processing model having shared layers and two or more task-specific layers, the shared layers comprising an encoder that has been trained with the two or more task-specific layers to map textual tokens into a vector space, wherein the encoder has been trained using at least two different task-specific sets of textual training data according to at least two different task- specific objectives and the two or more task-specific layers comprise at least two of a sentence classification layer, a text similarity layer, and a text classification layer; receiving input text; providing the input text to the multi-task natural language processing model;
obtaining a task-specific result produced by an individual task-specific layer of the multi-task natural language processing model; and performing a natural language processing operation based on the task- specific result.
Examiner notes that the:
RefDoc limitations anticipates the instant case because the RefDoc recites limitations are considered not patentably distinct and have overlapping scope; wherein the narrower scope anticipates the boarder scope limitations recited in the noted claim of the instant application.
Claim 24
A system comprising: a hardware processing unit; and a storage resource storing computer-readable instructions which, when executed by the hardware processing unit, cause the system to:
access a multi-task student instance, the multi-task student instance being a trained model that has been trained using first outputs of one or more first multi-task teacher instances to train a first task-specific layer of the multi-task student instance and using second outputs of one or more second multi-task teacher instances to train a second task- specific layer of the multi-task student instance; receive input data; provide the input data to the multi-task student instance; obtain a first task-specific result produced by the first task-specific layer of the multi-task student instance;
and output the first task-specific result, wherein the multi-task student instance, the one or more first multi-task teacher instances, and the one or more second multi-task teacher instances are instances of a multi-task machine learning model having one or more shared layers, the first task-specific layer, and the second task-specific layer.
25. (New) The system of claim 24, wherein the multi-task student instance is a natural language processing model and the input data comprises input text.
26. The system of claim 25, wherein the one or more shared layers include a lexicon encoder and a transformer encoder that have been trained to map textual tokens into a vector space.
Indication of Allowable Subject Matter
Claims 21-40 are indicated as allowable subject matter.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 21 recited the limitation “providing a multi-task machine learning model having one or more shared layers and two or more task-specific layers, … the one or more shared layers comprising an encoder configured to map natural language tokens into embeddings in a vector space; and performing … on the encoder and the two or more task-specific layers by using at least two different task-specific sets of textual training data to update respective parameters of the encoder according to at least two different task-specific objectives, wherein the two or more task-specific layers comprise at least two of a sentence classification layer, a text similarity layer, and a text classification layer.” has been searched, and to date no prior art has been found, thus the claim is indicated as directed to allowable subject matter pending resolution of the noted double patenting rejection above. Claims 32 and 40 recite similar limitations.
The closest prior art are cited above:
Ranjan et al (US 20190244014): teaches in [0025] that a general multitask learning framework for a deep convolutional neural network architecture has the lower layers shared among all the tasks and input domains. And multiple input domains (D.sub.1, D.sub.2, . . . D.sub.d) can be processed by the lower layers and can provide shared parameters θ.sub.s. Then, individual task-specific layers can further process and provide task specific parameters θ.sub.t1, θ.sub.t2 . . . θ.sub.ti.
Zou (US 20200065563): teaches in [0060] In the context of Deep Learning, there are typically two types of multi-task learning methods: hard parameter sharing and soft parameter sharing. With hard parameter sharing, the network shares its early layers for all tasks, and its later layers are task specific and isolated. This approach has a lower risk of overfitting, but needs human decisions about shared layers. And depicts in Fig 4 a hard parameter sharing for multi-task learning in deep neural networks, with the lower layers being shared and the upper layers being task-specific.
Meyerson et al. (US 11250314): teaches in 21:4-11: The system comprises an encoder generator. The encoder generator generates an encoder by accessing a set of processing submodules defined for the neural network-based system, constructing clones of the set of processing submodules, and arranging the clones in the encoder in a clone sequence starting from a lowest depth and continuing to a highest depth. The clones in the encoder are shared by a plurality of classification tasks…
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
McCann et al. (NPL: Learned in Translation: Contextualized Word Vectors): discloses in abstract: Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT)
to contextualize word vectors.
Zhao et al. (US 12182713): teaches in 3:48-55: The training data is input to a machine learning system to create a prediction model. In doing so, the training data is processed by an equidistant embedding system, a shared representation system, and an exclusive representation system. By analyzing features of the training data at various levels of abstraction, the prediction model can predict an outcome given a subsequent observation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWATOSIN ALABI whose telephone number is (571)272-0516. The examiner can normally be reached Monday-Friday, 8:00am-5:00pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWATOSIN ALABI/Primary Examiner, Art Unit 2129
Prosecution Insights