DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Using the subject matter eligibility test from page 74621 of the Federal Register Notice titled “2014 Interim Guidance on Patent Subject Matter Eligibility,” a two-step process is performed. Under step 1, the claims are analyzed to determine if the claim is directed to a process, machine, article of manufacture, or composition of matter. In this case, claims 1-17 are directed to a device, which is a machine or an article of manufacture. Step 2A (part 1 of the Mayo test), using the guidance from pages 50-57 of the Federal Register Vol. 84 No. 4 from Monday, January 7, 2019, requires applying a two-prong inquiry. In Prong One, examiners evaluate whether the claim recites a judicial exception, determining if the claim is directed to a law of nature, a natural phenomenon, or an abstract idea. In this case, claims 1 recites deriving a FAQ pair from speech data, which is a mental process. In Prong Two, examiners evaluate whether the judicial exception is integrated into a practical application that imposes a meaningful limit on the judicial exception. In this case, elements such as processor, memory, and neural network models are generic computing components, and do not integrate the abstract idea into a practical application.
Step 2B (part 2 of the Mayo test) requires analyzing the claims to determine if they recite additional elements that amount to significantly more than the judicial exception. In this case, the claims do not include additional elements that are sufficient to amount to significantly more than the abstract idea itself.
Regarding claim 1, deriving a FAQ pair from speech data is a mental process, which is an abstract idea. For example, a human could listen to speech data and create FAQ pairs from what they hear. Additional elements of processor, memory, and neural network models are generic computing components, and do not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claim 2, performing learning on the neural network model is a series of mathematical calculations, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 3, the loss calculations are mathematical calculations, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claims 4, 6-8, 10, 12-13, and 16-17, the limitations are further clarifications of the abstract idea.
Regarding claims 5 and 14, extracting feature vectors is a mental process or a mathematical calculation, and performing speech encoding is a mathematical calculation, which are abstract ideas. Inputting data to a model is mere extrasolution activity, and does not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claim 9, deriving a FAQ pair from speech data is a mental process, and calculating losses and preforming the learning are mathematical calculations, which are abstract ideas. Additional elements of processor, memory, and neural network models are generic computing components, and do not integrate the abstract idea into a practical application or constitute significantly more.
Regarding claim 11, performing training on the neural network model is a series of mathematical calculations, which is an abstract idea without integration into a practical application and without significantly more.
Regarding claim 15, obtaining preprocessed text data and performing text embedding are mental processes and mathematical calculations, which are abstract ideas. Inputting data to a model is mere extrasolution activity, and does not integrate the abstract idea into a practical application or constitute significantly more.
The limitations of the claims, taken alone, do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements individually. Applicable case law cited in the Federal Register includes, but is not limited to: Alice Corp., 134 S. Ct. at 2355-56, Digitech Image Tech., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344 (Fed. Cir. 2014), Benson, 409 U.S. at 63.
See "Preliminary Examination Instructions in view of the Supreme Court Decision in Alice Corporation Pty. Ltd. v. CLS Bank International, et al.," dated June 25, 2014, and the Federal Register notice titled "2014 Interim Guidance on Patent Subject Matter Eligibility" (79 FR 74618).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al. (US 2018/0082184 A1), hereinafter referred to as Guo, in view of Oktay et al. (US 2025/0173613 A1), hereinafter referred to as Oktay.
Regarding claim 1, Guo teaches:
An electronic device comprising:
a memory configured to store instructions (Fig. 2 element 204, para [0031], where memory is used); and
a processor electrically connected to the memory and configured to execute the instructions (Fig. 2 element 202, para [0031], where a processor is used),
wherein, when the instructions are executed by the processor, the processor is configured to perform a plurality of operations (para [0031], where computer programs are executed),
wherein the plurality of operations comprises deriving a frequently-asked-questions (FAQ) pair from speech data based on a neural network model trained in an end-to-end manner (para [0050], [0056-57], where answers are paired with user questions, para [0034], where the input is speech, and para [0024], where the neural conversational model is trained end-to-end), and
wherein the neural network model is based on a multi-modal language model (LM) capable of using text data and speech data simultaneously (para [0024], where a neural conversational model is used, and para [0034], [0043], where the model is multimodal, accepting speech and text input), to shift speech data, which is original data, to text data, which is augmented data.
Guo does not teach:
and contrastive learning is performed on the neural network model based on symmetric loss
Oktay teaches:
and contrastive learning is performed on the neural network model based on symmetric loss (para [0159], where a symmetric contrastive loss is used for learning)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Guo by using the training of Oktay (Oktay para [0159]) on the neural model of Guo (Guo para [0024]), by using contrastive learning, in order to provide initial pre-training without the need for deliberate, after-the-fact labelling (Oktay para [0010]).
Regarding claim 2, Guo in view of Oktay teaches:
The electronic device of claim 1, wherein multi-task learning, which uses the symmetric loss and cross-entropy loss simultaneously, is performed on the neural network model (Oktay para [0159], where the symmetric contrastive loss is used in training the CNN encoders).
Regarding claim 8, Guo in view of Oktay teaches:
The electronic device of claim 1, wherein the neural network model has one of preprocessed speech data or non-preprocessed speech data as an input (Guo para [0034], where the input is a voice input).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo, in view of Oktay, and further in view of Lee et al. (US 2021/0035562 A1), hereinafter referred to as Lee.
Regarding claim 5, Guo in view of Oktay teaches:
The electronic device of claim 1, wherein the deriving of the FAQ pair comprises:
Guo in view of Oktay does not teach:
extracting a feature vector of received speech data;
obtaining preprocessed speech data by performing speech encoding on the feature vector; and
inputting the preprocessed speech data to the neural network model.
Lee teaches:
extracting a feature vector of received speech data (para [0056-57], where a speech feature vector is obtained);
obtaining preprocessed speech data by performing speech encoding on the feature vector (para [0056-57], where encoding is performed on the feature vector); and
inputting the preprocessed speech data to the neural network model (para [0056-57], where the encoded speech vector is sent to the decoder RNN).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Guo in view of Oktay by using the preprocessing of Lee (Lee para [0056-57]) on the speech input of Guo in view of Oktay (Guo para [0034]), in order to remove noise and process the speech information to be in a form suitable to be input to the neural network (Lee para [0079]).
Allowable Subject Matter
Claims 9-17 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action.
Claims 3-4 and 6-7 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior art of Guo, Oktay, and Lee do not teach the limitations of the claims. Specifically, none of the cited prior art teaches the calculation of symmetric loss based on probability vectors that are intermediate outputs of the neural network model for each of speech and text data, calculating cross-entropy loss based on the FAQ pair that is a final output of the neural network model for speech data, or the specific structural layout and components in the neural network model, in combination with the other limitations of the claims. Hence, none of the cited prior art, either alone or in combination thereof, teaches the combination of limitations in the claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 10,929,392 B1 col. 7 line 58 – col. 8 line 2 teaches training an encoder-decoder model in an end-to-end fashion, while col. 8 line 56 – col. 9 line 16 teaches using the model for question answering.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRYAN S BLANKENAGEL whose telephone number is (571)270-0685. The examiner can normally be reached 8:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRYAN S BLANKENAGEL/Primary Examiner, Art Unit 2658
Prosecution Insights