Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Sun et al. (US 20250322170 A1), and in further view of Dande et al. (US 20250315720 A1) and Gupta (US 20150161386 A1).
With respect to claims 1 and 10 Sun teaches (claim 1) A computer-implemented method of performing natural language processing in a text corpus , comprising: (claim 10) A system for performing natural language processing in a text corpus, comprising: [[an input device configured to obtain a user-selected causal language modeling or a natural language processing task]]; a processor comprising a graphics processing unit (GPU) and connected to the input device ([0104] FIG. 7 illustrates a flowchart of an example method 700 for generating a merging instruction indicating whether to merge different data sets or portions thereof, in accordance with various embodiments. In certain embodiments, method 700 may be executed utilizing one or more processing devices (e.g., computing system 102 that may include hardware (e.g., a general purpose processor, a graphic processing unit (GPU)); and a memory connected to the processor ([0113] In particular embodiments, computer system 800 includes a processor 802, memory 804, storage 806, an input/output (I/O) interface 808, a communication interface 810, and a bus 812. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.); wherein the processor is configured to execute program instructions, comprising
receiving, by a processor, a block of words in the text corpus ([0042] FIG. 1 illustrates an example system 100 for generating a merging instruction indicating whether to merge different data sets or portions thereof, in accordance with various embodiments. System 100 may include a computing system 102, user devices 130-1 to 130-N (e.g., collectively referred to as “user devices 130”), databases 140 (e.g., training data database 142, model database 144, domain specific content database 146, word database 148, phrase database 152, rules database 154), or other components. In some embodiments, components of system 100 may communicate with one another using network 150, such as the Internet);
removing all vowels from the block of words with the processor to obtain a reduced text corpus having a reduced vocabulary (Sun ¶[0060] Data preprocessing subsystem 110 may be configured to remove, using vowel remover 230, vowels [vowel removal] from words 202 to create vowelless abbreviations 232. For example, the word “star” may have the vowelless abbreviation “str,” the word “database” may have the vowelless abbreviation “dtbs,” and the like. In some embodiments, data preprocessing subsystem 110 may be configured to store words 202 and vowelless abbreviations 232 in training data database 142 [reduced text corpus]. For example, the training data may comprise pairs of vowelless abbreviations and their corresponding words. In one or more examples, a label indicating a particular domain with which words 202 correspond may be stored with the training data, ¶ [0071] Different training data may be used to train different types of machine learning models. Furthermore, validation data may also be stored in training data database 142. The training data and the validation data may be identified and retrieved prior to first training process 300 beginning. In some embodiments, training data 302 may include features (e.g., n-grams of words) and labels (e.g., original words associated with the n-grams), or other information, or combinations thereof);
removing all vowels from each of the task-specific language data with the processor to obtain respective reduced task-specific language data Sun ¶[0060] Data preprocessing subsystem 110 may be configured to remove, using vowel remover 230, vowels [vowel removal] from words 202 to create vowelless abbreviations 232. For example, the word “star” may have the vowelless abbreviation “str,” the word “database” may have the vowelless abbreviation “dtbs,” and the like. In some embodiments, data preprocessing subsystem 110 may be configured to store words 202 and vowelless abbreviations 232 in training data database 142 [reduced text corpus].;
Sun doesn’t explicitly disclose however Dande teaches training a plurality of causal language models including a statistical model, a recurrent neural network (RNN)-based model, and a transformer-based model with a training portion of the reduced text corpus to obtain a plurality of respective working causal language models (Dande ¶[0101] As shown in block 408, the process further continues by integrating and training [models] multimodal AI models using the extracted features to develop a comprehensive understanding of the application workflows and decision-making criteria. It is understood that this may be executed by employing fusion-based machine learning techniques that combine features from different data types to train a cohesive model. For example, in some embodiments, the system may use ensemble learning methods where multiple models such as Recurrent Neural Networks (RNNs) for temporal data, Convolutional Neural Networks (CNNs) for visual data, and transformers for sequential text data are trained separately and their outputs are then combined, or fused, to make a final prediction [prediction is mapped to causal language models] or classification, ¶ [0084] The machine learning algorithms contemplated, described, and/or used herein include … a Bayesian method (e.g., naïve Bayes [statistical.);
receiving, by the processor, a plurality of task-specific language data in the text corpus ([0079] The data acquisition engine 202 may identify various internal and/or external data sources to generate, test, and/or integrate new features for training the machine learning model 224) Note: When models are used for classification the data is task-specific=classification;
training a plurality of natural language processing task models with the respective task data to obtain a plurality of working natural language processing task models (Dande ¶[0007] As such, embodiments of the invention relate to systems, methods, and computer program products for an integrated multimodal artificial intelligence framework for automated provisioning systems, the invention including: aggregating raw data from multiple data sources, wherein the data sources comprise logs, text, audio inputs, and visual inputs, resulting in aggregated raw data; producing a pre-processed dataset [for classification, dataset = task data] via normalizing and cleansing the aggregated raw data; determining extracted features from the pre-processed dataset using a combination of natural language processing for text data and computer vision for visual data,¶[0101] As shown in block 408, the process further continues by integrating and training multimodal AI models using the extracted features to develop a comprehensive understanding of the application workflows and decision-making criteria. It is understood that this may be executed by employing fusion-based machine learning techniques that combine features from different data types to train a cohesive model. For example, in some embodiments, the system may use ensemble learning methods where multiple models such as Recurrent Neural Networks (RNNs) for temporal data, Convolutional Neural Networks (CNNs) for visual data, and transformers [transformer, statistical] for sequential text data are trained separately and their outputs are then combined, or fused, to make a final prediction or classification. [classification is mapped to specific NLP task models] As such, in some embodiments, integration is performed within a development environment like Jupyter Notebook or PyCharm, utilizing machine learning frameworks such as TensorFlow or PyTorch, which support multimodal learning.);
Dande is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun further in view of Dande to training a plurality of natural language processing task models with the respective task data to obtain a plurality of working natural language processing task models. Motivation to do allow reducing the potential for errors and improve efficiency over time (Dande [0006]).
None of Sun and Dande explicitly disclose however Gupta teaches an input device configured to obtain a user-selected causal language modeling or a natural language processing task(Gupta ¶Claim 17. A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor to perform operations for generating data models in a mobile device [input device], the operations comprising: receiving a full classifier model that includes a plurality of test conditions; identifying mobile device features used by one of: a software application of the mobile device; and a type of software application that may execute on the mobile device; identifying test conditions in the plurality of test conditions that evaluate the identified mobile device features; generating an application-based classifier model that prioritizes the identified test conditions, the application-based classifier model being selected [selecting] from a group consisting of: an application-specific classifier model; and an application-type-specific classifier model; and using [performing] the generated application-based classifier model to classify a behavior of the mobile device.;
selecting causal language modeling or a natural language processing task, each having a respective one of the plurality of the working causal language models or the plurality of working natural language processing task models (Gupta ¶Claim 17. A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor to perform operations for generating data models in a mobile device [user device], the operations comprising: receiving a full classifier model that includes a plurality of test conditions; identifying mobile device features used by one of: a software application of the mobile device; and a type of software application that may execute on the mobile device; identifying test conditions in the plurality of test conditions that evaluate the identified mobile device features; generating an application-based classifier model that prioritizes the identified test conditions, the application-based classifier model being selected [selecting] from a group consisting of: an application-specific classifier model; and an application-type-specific classifier model; and using [performing] the generated application-based classifier model to classify a behavior of the mobile device.); and
performing the selected causal language modeling or the selected natural language processing task using the respective working causal language model or working natural language processing task model with the processor (Gupta ¶Claim 17. A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor to perform operations for generating data models in a mobile device, the operations comprising: receiving a full classifier model that includes a plurality of test conditions; identifying mobile device features used by one of: a software application of the mobile device; and a type of software application that may execute on the mobile device; identifying test conditions in the plurality of test conditions that evaluate the identified mobile device features; generating an application-based classifier model that prioritizes the identified test conditions, the application-based classifier model being selected [selecting] from a group consisting of: an application-specific classifier model; and an application-type-specific classifier model; and using [performing] the generated application-based classifier model to classify a behavior of the mobile device).
Gupta is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun as modified above, further in view of Gupta to select causal language modeling or a natural language processing task. Motivation to do so would allow to efficiently generate models (Gupta [0094]).
Claims 2, 11, 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande Gupta in further view of Wang (US 20220147715 A1).
With respect to claims 2 and 11 none of Sun, Dande and Gupta explicitly disclose however Wang teaches wherein the plurality of natural language processing tasks is selected from the group consisting of a text classification task, a sequence labeling task, and a translation task ([0100] The processor in FIG. 1 and FIG. 2 may perform data training/machine learning/deep learning by using a neural network model or another model (for example, a support vector machine-based model), and execute a natural language processing application (for example, text classification, sequence labeling, reading comprehension, text generation, text inference, translation) on a text sequence by using a model obtained through final data training or learning, to obtain a corresponding processing result, ¶ [0112] (4) A recurrent neural network (recurrent neural network, RNN) is used to process sequence data.).
Wang is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Wang to select from the group consisting of a text classification task, a sequence labeling task, and a translation task. Motivation to do so would be too improve a capability of understanding by a model, and improve accuracy of a result of processing a target task (Wang [0006]).
With respect to claims 5 and 14 Wang further teaches wherein the working natural language processing task model is a RNN-based model when the selected task is the sequence labeling task, and wherein the performing the selected task further comprises assigning a label to a sequence of tokens in the block of words to obtain a labeled sequence of tokens ([0100] The processor in FIG. 1 and FIG. 2 may perform data training/machine learning/deep learning by using a neural network model or another model (for example, a support vector machine-based model), and execute a natural language processing application (for example, text classification, sequence labeling, reading comprehension, text generation, text inference, translation) on a text sequence by using a model obtained through final data training or learning, to obtain a corresponding processing result.).
Claims 3 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande Gupta in further view of Hargrave (US 5724593 A)
With respect to claims 2 none of Sun, Dande and Gupta explicitly disclose however Hargrave teaches wherein the statistical model is a n-gram language model, wherein the n-gram language model is selected from the group consisting of 2-gram language model, 3-gram language model, 4-gram language model, 5-gram language model, and 6-gram language model (Col6ll29-44 Moving now to step 109, each text segment in the text sample is selected and processed through steps 109-117 in turn. The selected text segment is first tokenized in step 109. Tokenizing step 109 generates a set of letter n-grams included in the selected text segment. In a preferred embodiment, trigrams (i.e., three sequential characters) are used for English and Indo-European languages while bigrams (i.e., two sequential characters) are used for Asian languages such as Korean, Japanese and Chinese. It is expressly understood that the size of the n-gram is not a limitation of the present invention. Any n-gram size can be chosen including 1-grams, 2-grams, 3-grams, 4-grams, 5-grams, 6-grams, or higher. Various n-gram sizes will prove useful in some applications. It is also contemplated that a single translation memory will use more than one n-gram size. N-grams can also be chosen to approximate syllables in the source language).
Hargrave is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Wang wherein the n-gram language model is selected from the group consisting of 2-gram language model, 3-gram language model, 4-gram language model, 5-gram language model, and 6-gram language model. Motivation to do so would be too improve a capability of understanding by a model, and improve accuracy of a result of processing a target task (Wang [0006]).
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande, Gupta and Wang in further view of Glottmann (US 20200043600 A1).
With respect to claims 4 none of Sun, Dande, Gupta and Wang explicitly disclose however Glottmann teaches wherein the working natural language processing task model is a RNN-based model when the selected task is the text classification task, wherein the text classification task comprises a binary sentiment analysis, and wherein the performing the selected task further comprises categorizing the block of words into one of two subcategories ([0056] In some embodiments, the NLP servers 127 may include RNNs configured to provide sentiment analysis. In particular, the RNNs may be configured to classify text as having positive or negative sentiment. For example, the RNNs may indicate whether the radiologists positively or negatively reported liver lesions in the radiology report).
Glottmann is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Glottmann to categorize the block of words into one of two subcategories. Motivation is to increase efficiency by using binary classification ([0097] Glottmann).
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande, Gupta and Wang in further view of Fan (US 20230360636 A1).
With respect to claims 6 and 15 none of Sun, Dande, Gupta and Wang explicitly disclose however Fan teaches wherein the working natural language processing task model is a transformer-based model when the selected task is the translation task (Fan ¶[0022] It is known to persons skilled in the art that transformer learning models may be implemented based on an encoder-decoder architecture such as BERT as mentioned above, including an encoder and a decoder each made up of multi-head attention layers and feedforward layers. For example, transformer learning models may be implemented for linguistic translation or prediction tasks, wherein a sequence of word tokens is input into the transformer learning model and a translated or predicted sequence of word tokens is output from the transformer learning model,¶[0049] In the computing of an ASR task by a transformer learning model, a sample of input audio signals may be tokenized into a sequence made up of a plurality of audio tokens. In FIG. 2, the audio signal, illustrated by a spectrogram 202 in FIG. 2, is convoluted through a convolutional input layer 204 to tokenize the audio signal), and wherein the performing the selected task further comprises: tokenizing each reduced text in the reduced texts corpus (Fan ¶[0022] It is known to persons skilled in the art that transformer learning models may be implemented based on an encoder-decoder architecture such as BERT as mentioned above, including an encoder and a decoder each made up of multi-head attention layers and feedforward layers. For example, transformer learning models may be implemented for linguistic translation or prediction tasks, wherein a sequence of word tokens is input into the transformer learning model and a translated or predicted sequence of word tokens is output from the transformer learning model,¶[0049] In the computing of an ASR task by a transformer learning model, a sample of input audio signals may be tokenized into a sequence made up of a plurality of audio tokens. In FIG. 2, the audio signal, illustrated by a spectrogram 202 in FIG. 2, is convoluted through a convolutional input layer 204 to tokenize the audio signal); and
translating the reduced texts corpus in a first language to a second language to obtain a translated texts corpus (Fan ¶[0022] It is known to persons skilled in the art that transformer learning models may be implemented based on an encoder-decoder architecture such as BERT as mentioned above, including an encoder and a decoder each made up of multi-head attention layers and feedforward layers. For example, transformer learning models may be implemented for linguistic translation or prediction tasks, wherein a sequence of word tokens is input into the transformer learning model and a translated or predicted sequence of word tokens is output from the transformer learning model,¶[0049] In the computing of an ASR task by a transformer learning model, a sample of input audio signals may be tokenized into a sequence made up of a plurality of audio tokens. In FIG. 2, the audio signal, illustrated by a spectrogram 202 in FIG. 2, is convoluted through a convolutional input layer 204 to tokenize the audio signal)
Fan is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Fan to wherein the working natural language processing task model is a transformer-based model when the selected task is the translation task. Motivation is to improve accuracy by evaluation error rate of transfer model by fine-tuning ([0029] Fan)
.
Claims 7, 8, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande, Gupta in further view of Kanzelberg (US 20120278315 A1).
With respect to claims 7 and 16 none of Sun, Dande, Gupta and Wang explicitly disclose however Kanzelberg teaches wherein the removing all vowels from the block of words comprises: parsing the block of words in the text corpus to identify one or more vowels in the block of words (Kanzelberg ¶[0044] For instance, consider the strings Q and R and their phonetic transductions Q' and R'. If the strings Q and R are compared, portions 402 of Q and R are matching. If the phonetic transductions Q' and R' are compared, matching phonetic 7-graphs V@D@F@N (the "@" being a generic character replacing any vowel) are detected); and
replacing the one or more vowels with a mask-symbol, wherein the mask-symbol corresponds to each vowel of the one or more vowels. wherein a total number of each word in the block of words is preserved (Kanzelberg ¶[0044] For instance, consider the strings Q and R and their phonetic transductions Q' and R'. If the strings Q and R are compared, portions 402 of Q and R are matching. If the phonetic transductions Q' and R' are compared, matching phonetic 7-graphs V@D@F@N (the "@" being a generic character replacing any vowel) are detected)
Kanzelberg et al. show “@” symbol above. They do not specifically show ‘#” symbol can be used for vowels. It would be obvious to one of ordinary skill in the art at the time of the invention use “#” as needed because it allows suppression of vowels in compatible ways and is a matter of design choice.
Kanzelberg is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Kanzelberg to replacing the one or more vowels with a mask-symbol. Motivation is to reduce noise and improve normalization in text processing
With respect to claims 8 and 17 Kanzelberg further teaches wherein the step of removing all vowels includes replacing each vowel with the mask-symbol that is a single character including a “#” (Kanzelberg ¶[0044] For instance, consider the strings Q and R and their phonetic transductions Q' and R'. If the strings Q and R are compared, portions 402 of Q and R are matching. If the phonetic transductions Q' and R' are compared, matching phonetic 7-graphs V@D@F@N (the "@" being a generic character replacing any vowel) are detected).
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sun, Dande, and Gupta in further view of Saeed (US 11251031 B1) and Weston (US 20250054493 A1).
With respect to claims 9 and 18 none of Sun, Dande, Gupta explicitly disclose however Saeed teaches further comprising: retrieving the block of words from the reduced text corpus using the RNN language model, wherein the RNN language model consists of two layers of bidirectional LSTM each having 512 hidden units and an embedding layer with a dropout of 0.25 to obtain a retrieved text corpus; (Saeed ¶Col2ll32-59Each negative pair can be selected via an online hardest negative mining process, in which negative spectra and peptides that are closest to Q and P are selected for a given batch after each forward pass. The PSN can further comprise an embedding layer before the Bi-LSTM [bi-directional LSTM] layer, and the embedding layer can use a vocabulary size of 20 or 30 to construct embeddings. The Bi-LSTM layer can have a hidden dimension of 512, [512] and the two fully connected layers of the PSN can be after the Bi-LSTM layer... The SSN can further comprise a dropout [dropout] mechanism with a probability of 0.3 after a first layer of the two fully connected hidden layers of the SSN and before a second layer of the two fully connected hidden layers of the SSN.
Saeed et al. show a dropout of 0.3. They do not specifically show thar 0.25 can be used for dropout. It would be obvious to one of ordinary skill in the art at the time of the invention use 0.25 as needed because it allows for dropout in compatible ways and is a matter of design choice.
Saeed is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Saeed wherein the RNN language model consists of two layers of bidirectional LSTM each having 512 hidden units and an embedding layer with a dropout of 0.25 to obtain a retrieved text corpus. Motivation is to reduce prevent overfitting and improve generalization.
None of Sun, Dande, Gupta and Saeed explicitly disclose however Weston teaches and performing a post-processing including a spelling correction and a grammar correction to refine the retrieved text corpus ([0115] In some implementations, before the generated response is sent to the user, it may undergo a post-processing step. Post-processing may involve tasks like correcting grammar and spelling, ensuring the use of appropriate language, or adding any system-specific formatting. )
Weston is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun, as modified above, further in view of Weston performing a post-processing including a spelling correction and a grammar correction to refine the retrieved text corpus. Motivation is to ls to improve the accuracy or completeness of their predictions (Weston, [0059]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ATHAR N PASHA whose telephone number is (408)918-7675. The examiner can normally be reached Monday-Thursday Alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached on (571)272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ATHAR N PASHA/ Primary Examiner, Art Unit 2657
Prosecution Insights