DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1 and 11 are amended. Claims 1-3, 6-13, and 16-20 are presented for examination.
Response to Arguments
Rejection under 35 U.S.C. 103
Applicant's arguments filed have been fully considered but they are not persuasive. Applicant argues, “Specifically, the claims recite ‘wherein the electronic device comprises a single mobile device, the AI model and a server implemented on the single mobile device, and the method is performed entirely on the single mobile device.’ This is not disclosed by the cited references either separately or in combination. Fox is directed towards a client server model and fails to disclose or suggest at least the limitations at issue.” Applicant also argues, “As described in Fox, FIG. 4 is the architecture that is used for two separate devices; both the client computing device 110 and the server computer 120 (which are illustrated in FIG. 1).”
However, Fig. 4 of Fox depicts the client computing device 110 and server computer 120 working in combination as a single device rather than as two separate devices. For example, Fox recites communicating with different hardware components within a system, “[0062] Communications fabric 404 can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications, and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system.” This system, as depicted in Fig. 4 of Fox contains a display and I/O interface, representing the client device, “[0067] I/O interface(s) 406 allows for input and output of data with other devices that may be connected to client computing device 110.” These components of the client device are connected together as hardware with components of the server to form a single device, which can be a mobile device. Therefore, rather than two separate devices, Fig. 4 of Fox shows an embodiment where the client and server are connected together in a single device to perform the functions described in Fox. Further, Fox describes an embodiment including all components of the server computer as residing on the client device. Specifically, Fox recites, “[0025] Database 122 is a repository for data used by program 150. In the depicted embodiment, database 122 resides on server computer 120. In another embodiment, database 122 may reside on client computing device 110…” and “[0034] In various embodiments, client versions of program 150 resides on client computing device 110…” The database and program, representing the components of the server computer as shown in Fig. 1 of Fox, can be implemented on a single mobile client device, such as a smart phone as described in paragraph [0021] of Fox. Thus, Fox teaches using a single mobile device to perform the cited limitations of the claim.
Applicant further argues, “The Office Action is improperly picking words from paragraphs [0031]-[0032] (neural network algorithms and models) to assert that the ‘special purpose computer’ forty-three paragraphs later is an NPU. Applicant notes that paragraph [0029] of Fox mentions statistical models - perhaps the ‘special purpose computer’ is optimized for statistical models…. Thus, assuming that a mention of a ‘special purpose computer’ in paragraph [0075], forty-three paragraphs after a mention of neural networks in paragraph [0032], is an NPU, is not a reasonable assumption.” Applicant also argues, “Fox neither explicitly discloses nor implicitly suggests an NPU (Neural Processing Unit), which is a dedicated hardware specialized for AI computation. In particular, an NPU is a dedicated processor designed to perform deep learning-based neural network computations at high speed and is technically distinct from a general-purpose computer. Accordingly, it is difficult to regard the NPU-based AI processing hardware configuration of the present claims as being obvious to a person skilled in the art from Fox.”
However, it would be reasonable and obvious to a person of ordinary skill in the art to use an NPU to perform the functions described in Fox. The corpus link model (CLM) is one of the main features of the invention described in Fox and can be implemented as a variety of neural network types and architectures, which Fox describes, “[0031-0032] In various embodiments, CLM 126 utilizes transferrable neural network algorithms and models (e.g., long short-term memory (LSTM), deep stacking network (DSN), deep belief network (DBN), convolutional neural networks (CNN), compound hierarchical deep models, etc.) that can be trained with supervised or unsupervised methods. In various embodiments, CLM 126 is a simple neural network… CLM 126 contains one or more recurrent neural networks (RNNs).” Therefore, it is reasonable to assume that the “special-purpose computer” recited in paragraphs [0075] and [0077] of Fox can be a NPU to aid in the performance of the neural networks described in Fox rather than for optimizing statistical models, which is rarely mentioned in Fox. Additionally, NPUs were first introduced in 2017 specifically for use in mobile devices. Thus, it would be obvious to person of ordinary skill in the art at the time to use an NPU as a “special purpose computer” in the form of processing hardware for the purpose of the functions described in Fox.
Applicant further argues, “Fox neither explicitly discloses nor suggests that an artificial intelligence model is trained using a corpus generated based on a learning text in which collocations are merged, that the merging of collocations is performed based on the co- occurrence frequency of words included in the learning text, or that a collocation having the highest consecutive co-occurrence frequency is preferentially merged into a single collocation.”
However, Fox discloses training the CLM, a neural network, on labeled collocated term pairs from a corpus. Specifically, Fox recites, “[0044] program 150 performs supervised training with the labeled vectorized data, as described in step 202. For example, program 150 feeds query/collocated term pairs into CLM 126, allowing program 150 to make inferences between the query term data and collocated term data (i.e., label). In an embodiment, program 150 trains CLM 126 with a plurality of feature vectors originating from data extracted from related queries, topics, communications, or author specific discussions or queries located in linguistic corpus 124…” Further, the Fox reference is combined with the Wang reference in order to teach a corpus in which collocations are merged based on co-occurrence of words. Specifically, Wang teaches, “[3.1] Measure Word Frequency. It refers to the number of co-occurrences between the measure word m and concept c, and is denoted mfm,c. To get the measure word frequency, we need to identify the measure word collocating with the concept in certain context… co-occurrence frequency can be counted based on the identification results for each pair of the concept and measure word.”
Applicant further argues, “The Office Action asserts that U.S. Patent Application Publication No. 2023/0065965 to Liu et al. (hereinafter Liu) discloses the NPU in paragraph [00147]. Applicant respectfully disagrees. Liu discloses training device as separate from the client device.”
However, the Liu reference teaches using an NPU in either a execution device or training device, “[0147] The chip includes a neural processing unit (NPU) 40. The chip may be disposed in the execution device 110 shown in FIG. 4, and configured to complete calculation work of the calculation module 111. Alternatively, the chip may be disposed in the training device 120 shown in FIG. 4, and configured to complete training work of the training device 120 and output the target model/rule 101.” The Liu reference is used in combination with the Fox reference in order to teach a preprocessing operation of the NPU in the single mobile device described in the Fox reference.
Applicant further argues, “Farhan does not disclose or suggest generating information about a base language (i.e., a language to be translated) based on the pre-determined rule and applying the generated information to the AI model. Rather, the disclosures of paragraphs [0010], [0067], and [0115] relate to (i) identification of languages and creation of a word-vector mapping model, (ii) vector conversion using a trained sequence-to-sequence converter, and (iii) statistical training of neural network parameters using sentence pairs. These disclosures concern optimization and utilization of internal vector representations and neural network weights for translation.”
However, Farhan discloses generating information, such as vector values, about a base language using contextual meaning of the words in a specific language using a word-vector mapping model, which can act as a rule. The AI model is the multilingual translation model described in Farhan. Specifically, Farhan recites, “[0053] multilingual translation device uses a multilingual translation model to identify a language corresponding to each word included in a multilingual sentence and to obtain a vector value corresponding to each word included in the multilingual sentence, based on the identified language. In one embodiment of the disclosure, the language corresponding to each word included in the multilingual sentence may be identified based on a contextual meaning of each word of the multilingual sentence. The word-vector mapping model is a model mapping words to vector values corresponding thereto, in consideration of a language corresponding to each word.” Additionally, Farhan describes a sequence-to-sequence converter as part of the multilingual translation model, “[0055] the multilingual translation model may include a sequence-to-sequence converter, and the multilingual translation device may convert the vector values corresponding to the words of the multilingual sentence into vector values corresponding to the target language by using the sequence-to-sequence converter.” Farhan teaches training the sequence-to-sequence converter, which is part of the multilingual translation model, using the vector values, “[0115] the multilingual translation device may train the sequence-to-sequence converter by using an LSTM algorithm. The multilingual translation device may train the relationship between the arrangement of the vector values corresponding to the source reference sentence and the arrangement of the vector values corresponding to the target reference sentence through an artificial neuron network.” Thus, Farhan describes generating information about a base language using contextual meaning and a word-vector mapping model as a rule, and applying that generated information to an AI model in the form of the multilingual translation model.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6, 9-11, 16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Farhan et al. (US 20190332677 A1; hereinafter referred to as Farhan) in view of Fox et al. (US 20210073336 A1; hereinafter referred to as Fox) and Wang et al. (Wang, M., Zhu, H., & Yu, S. (2008, December). Concept Acquisition from Corpora: Using an Automatic Clustering Method Based on Chinese Measure Words. In 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (Vol. 3, pp. 295 - 298). IEEE.; hereinafter referred to as Wang).
Regarding claim 1, Farhan discloses: a method of controlling an electronic device, the method comprising: receiving, by the electronic device, an input of a user regarding first text including a plurality of languages ([0047] the multilingual translation device may receive the multilingual sentence from a terminal of a user based on user input), the user input identifying a base language into which the first text is to be translated ([0013] by executing the one or more instructions, set languages into which translation is to take place, based on a user input, and create the word-vector mapping model based on the set languages), wherein the base language is a language of the plurality of languages of the first text ([0072] The multilingual translation device may receive an input for setting a target language from the user, and set a target language for translation, based on the received input. The multilingual translation device may receive the input for setting a target language from the user via the window 420 for setting a target language. Also see Fig. 4, where the target language can be a language of the input text.);
identifying, by one or more processors of the electronic device, the first text ([0047] the multilingual translation device may extract the multilingual sentence from text contained in data, which is stored in the multilingual translation device or created by the multilingual translation device);
applying, by the one or more processors of the electronic device, the first text to an artificial intelligence (Al) model ([0115] The multilingual translation device may train the relationship between the arrangement of the vector values corresponding to the source reference sentence and the arrangement of the vector values corresponding to the target reference sentence through an artificial neuron) trained based on a corpus in which words of the plurality of languages are mapped to each other ([0052] the word-vector mapping model may be a model mapping corresponding vector values to words included in a first sentence written in a single language and words included in second sentences obtained by translating at least one of the words of the first sentence into at least one different language. Also see Fig. 7.);
generating information about the base language ([0067] and obtaining vector values corresponding to words included in the multilingual sentence by using a multilingual translation model, converting the obtained vector values into vector values corresponding to a target language, and obtaining a sentence in the target language, based on the vector values corresponding to the target language) based on a pre-determined rule ([0010] identify languages corresponding to the words of the first sentence and the words of the second sentences, and create the word-vector mapping model by mapping the corresponding vector values of the words of the first sentence and the words of the second sentences, based on the identified languages) and applying the generated information ([0055] the multilingual translation model may include a word-vector mapping model, and the multilingual translation device may obtain the vector values corresponding to the words included in the multilingual sentence by using the word-vector mapping model) to the Al model ([0115] the multilingual translation device may train the sequence-to-sequence converter by using an LSTM algorithm. The multilingual translation device may train the relationship between the arrangement of the vector values corresponding to the source reference sentence and the arrangement of the vector values corresponding to the target reference sentence through an artificial neuron network. The multilingual translation device may obtain an optimum weight for outputting a target sentence, when a source sentence is input, by inputting a large number of pairs of input sentences and output sentences to the artificial neuron network.);
identifying, by the one or more processors of the electronic device, second text corresponding to the first text from the Al model ([0094] the multilingual translation device creates second sentences by translating at least one word included in a first sentence among the sentences into at least one different language), the second text including a first single language from among the plurality of languages ([0073] multilingual translation device may translate the received sentences into the target language by using a multilingual translation model. The multilingual translation device may display results of translating the received sentences on the window 430 for displaying a translation result);
and displaying on a display of the electronic device, by the electronic device, the second text... ([0070] Referring to FIG. 4, a multilingual translation device may include a display, and may cause the display to display a user interface 400 for multilingual translation on the display. The user interface 400 may include a window 410 for receiving a sentence to be translated, a window 420 for setting a target language, and a window 430 for displaying a translation result).
Farhan does not explicitly, but Fox teaches: wherein the one or more processors include at least one neural processing unit (NPU) designed in a hardware structure ([0075] computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The special purpose computer can be a computer specialized for processing Al and ML tasks.) specialized for processing the Al model ([0077] It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions),
wherein the Al model is trained by using a corpus ([0028] linguistic corpus 124 includes collections of queries (e.g., associated topics), collocated and colligated term pairs (e.g., additional search and query terms). Each pair includes a query and a corresponding collocated and colligated term or sequence. A query term or sequence may be a textual term or sequence, in a natural language or a computer-generated representation. For example, the query (e.g., topic) “allergy” is paired with the collocated and colligated terms “acute” and “concerns” and forming the complete query sequence/phrase “acute allergy concerns.”) generated based on training text ([0044] program 150 performs supervised training with the labeled vectorized data, as described in step 202. For example, program 150 feeds query/collocated term pairs into CLM 126, allowing program 150 to make inferences between the query term data and collocated term data (i.e., label)) in which collocations are merged... ([0029] Corpus link model (CLM) 126, hereinafter CLM 126, contains one or more models, containers, documents, sub-documents, matrices, vectors, and associated data, modeling one or more feature sets, such as results from linguistic analysis. In an embodiment, linguistic analysis determines query characterizations and representations, collocated term consolidation),
wherein the method further includes displaying, on the display of the electronic device, the merged collocations in accordance with the pre-determined rule ([0007] one or more computer processors display the one or more generated collocated terms according to the generated co-occurrence rating of each collocated term. Embodiments of the present invention utilize trained models based on specific corpuses to generate generalized collocated terms, allowing the presentation of an optimal set of linked terms within the construct of a collaborative chat search) and wherein the electronic device comprises a single mobile device ([0034] client versions of program 150 resides on client computing device 110 and/or any other computing device (not depicted) within computational environment 100), the AI model ([0044] Program 150 trains corpus link model (step 204). Program 150 trains one or more models contained in CLM 126. In an embodiment, program 150 initializes CLM 126 with randomly generated weights) and a server ([0062, 0064] Server computer 120 includes communications fabric 404, which provides communications between cache 403, memory 402, persistent storage 405, communications unit 407, and input/output (I/O) interface(s) 406… Program 150 may be stored in persistent storage 405 and in memory 402 for execution), implemented on the single mobile device, and the method is performed entirely on the single mobile device ([0024] Server computer 120 can be a standalone computing device, a management server, a web server, a mobile computing device, or any other electronic device or computing system capable of receiving, sending, and processing data… server computer 120 may contain other applications, databases, programs, etc. which have not been depicted in computational environment 100. Server computer 120 may include internal and external hardware components, as depicted and described in further detail with respect to FIG. 4. Fig 4 describes an embodiment where the client and server are a single device.).
Farhan and Fox are considered analogous in the field of language processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan to combine the teachings of Fox because doing so would allow for collocations from different languages to be more efficiently and accurately determined by using linguistic analysis and a specialized processor (Fox [0017] Embodiments of the present invention recognize that system efficiency (e.g., reduction in system processing requirements such as memory and central processing unit utilization) can be improved by eliminating the generation of unlikely or improbable collocated terms).
The combination of Farhan and Fox does not explicitly, but Wang teaches: the merging of the collocations is performed based on the number of co-occurrences of words included in the training text ([3.1] Measure Word Frequency. It refers to the number of co-occurrences between the measure word m and concept c, and is denoted mfm,c. To get the measure word frequency, we need to identify the measure word collocating with the concept in certain context... co-occurrence frequency can be counted based on the identification results for each pair of the concept and measure word), and collocations having a highest number of consecutive occurrences ([4.1] After the pre-processing of noun phrases (including measure words) identification mentioned in 3.1, measure word frequency can be counted from the corpus. And we get 194 different measure words that collocate with all the concepts) are preferentially merged into single collocations... ([4.3] Take class ‘person’, ‘assets’ and ‘event’ for example, ‘person’ concepts often collocate with ‘wei’ and ‘ming’, such as ‘a businessman’ and ‘an athlete’. Also see section 4.3.)
Farhan, Fox, and Wang are considered analogous in the field of language processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan and Fox to combine the teachings of Wang because doing so would provide the concept of determining collocations of words in a particular language and merging the collocations in order to better understand the translation of the word in another language, leading to improved translations and collocation determinations (Wang [Section 2] each measure word can only collocate with certain classes of nouns which share similar semantic properties denoted by the measure word; instead each noun has its associated measure words which carry out different information. Since measure words indicate some salient semantic properties of nouns, the combination of a noun's associated measure words can describe the semantic content of the noun to some extent).
Regarding claim 6, the combination of Farhan, Fox, and Wang teaches: the method of claim 1. Farhan further teaches: wherein the Al model is trained with second training text generated by replacing a first word of a first language included in first training text with a second word of a second language, based on the corpus ([0106] the multilingual translation device may train the word embedding algorithm by using the first sentence 811 “I'm going to school” and second sentences 821 “I'm going to école”, “I'm going to colegio”, and “I'm going to scuola”. In this case, the contextual meaning of the anchor word “school” in the first sentence 811 and the contextual meanings of “école”, “colegio”, and “scuola”, which are translated from “school”, in the second sentences 821 are the same).
Regarding claim 9, the combination of Farhan, Fox, and Wang teaches: the method of claim 6. Farhan further teaches: wherein the Al model is trained with the first training text comprising the single language and the second training text comprising the plurality of languages ([0106] the multilingual translation device may train the word embedding algorithm by
using the first sentence 811 “I'm going to school” and second sentences 821 “I'm going to école”, “I'm going to colegio”, and “I'm going to scuola”).
Regarding claim 10, the combination of Farhan, Fox, and Wang teaches: the method of claim 1. Farhan further teaches: a non-transitory computer-readable recording medium having recorded thereon a computer program including instructions which, when executed, cause an electronic device to perform ([0137] A method of operating the multilingual translation device 1100 may be recorded on a non-transitory computer-readable recording medium storing one or more programs including instructions for performing the method).
Regarding claim 11, Farhan teaches: an electronic device, the electronic device comprising: a display; a user interface device configured to receive an input of a user... ([0070] a multilingual translation device may include a display, and may cause the display to display a user interface 400 for multilingual translation on the display). The rest of the claim recites similar limitations as claim 1 and therefore is rejected similarly.
Regarding claim 16, the claim recites similar limitations as claim 6 and therefore is rejected similarly.
Regarding claim 19, the claim recites similar limitations as claim 9 and therefore is rejected similarly.
Claims 2-3 and 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Farhan in view of Fox and Wang, as applied to claims 1, 6, 9-11, 16, and 19 above, and further in view of Shazeer et al. (US 20170228414 A1; hereinafter referred to as Shazeer).
Regarding claim 2, the combination of Farhan, Fox, and Wang teaches: the method of claim 1. The combination of Farhan, Fox, and Wang does not explicitly, but Shazeer teaches: wherein the Al model is trained ([0045] the generated low dimensional feature embedding vectors 108 may be stored and/or provided for use in a natural language processing system, image Classification system, or other machine learning model) by using the corpus generated based on the words of the plurality of languages ([0038] Example features include words from a particular language, strings of words from a particular language, or syntactic paths between words. For example, the methods and systems may be used to generate (or “train’) low dimensional vector representations of words which may in turn be used to perform natural language processing tasks, including determining semantic similarity, parsing or translation. Co - occurrences of such features may include how many times a word or string of words appear directly next to another word or string of words. Translation indicates a plurality of languages.) which co-occur from training texts comprising the plurality of languages ([0065] if the m row features are words in a particular language and then column features are strings of words, the entries of the mxn feature co-occurrence matrix may represent how many times the m words and n strings of words occur together in the corpus of text).
Farhan, Fox, Wang, and Shazeer are considered analogous in the field of language processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan, Fox, and Wang to combine the teachings of Shazeer because doing so would allow for co-occurrences of words to be more closely analyzed to improve language translation (Shazeer [0025] a system for generating feature embeddings from a feature co- occurrence matrix, as described in this specification, generates feature embeddings based on both observed co-occurrences and unobserved co-occurrences. This enables the system to identify anti-associations between features—increasing the amount of relevant information used to generate the feature embeddings and thus improving the accuracy and completeness of the generated feature embeddings).
Regarding claim 3, the combination of Farhan, Fox, Wang, and Shazeer teaches: the method of claim 2. Shazeer further teaches: wherein the Al model is trained by using the corpus generated by mapping words included in the training texts in an order of frequency of co- occurrence ([0014] the initial feature co-occurrence matrix includes m row features and n column features; sorting the m rows into descending order of feature frequency and generating m/k row blocks by collecting the m sorted rows into k-element row blocks, wherein k is chosen based on the target dimensionality d and a desired computational efficiency; sorting then columns into descending order of feature frequency. The feature frequency can be word frequency.).
Regarding claim 12, it recites similar limitations as claim 2 and therefore is rejected similarly.
Regarding claim 13, it recites similar limitations as claim 3 and therefore is rejected similarly.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Farhan in view of Fox and Wang, as applied to claims 1, 6, 9-11, 16, and 19 above, and further in view of Altschul et al. (US 20220277149 A1; hereinafter referred to as Altschul).
Regarding claim 7, the combination of Farhan, Fox, Wang teaches: the method of claim 6. Farhan further teaches: wherein the second training text is generated by replacing the first word with the second word mapped to have a meaning similar to a meaning of the first word, based on the corpus ([0106] the multilingual translation device may train the word embedding algorithm by using the first sentence 811 “I’m going to school” and second sentences 821 “I'm going to école”, “I'm going to colegio”, and “I'm going to scuola”. In this case, the contextual meaning of the anchor word “school” in the first sentence 811 and the contextual meanings of “école”, “colegio”, and “scuola”, which are translated from “school”, in the second sentences 821 are the same).
The combination of Farhan, Fox, and Wang does not explicitly, but Altschul teaches: wherein the Al model is trained with the second training text so that a loss between the first training text and third training text output from the Al model in response to application of the second training text to the Al model decreases ([0056] Statistical language models (e.g., transformer language models and autoregressive language models) may be trained from a training corpus, such as conversations data store 410, using any appropriate techniques. Training techniques may include supervised training or unsupervised training. The training of a language model may comprise multiple rounds where each round updates the parameters of the language model by minimizing a loss function).
Farhan, Fox, Wang, and Altschul are considered analogous in the field of language processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan, Fox, and Wang to combine the teachings of Altschul because doing so would provide more diverse training data that includes language used in different scenarios (Altschul [0028] A language mode! may be trained from a corpus of training data. Since different people in different scenarios may have conversations about different subject matters and using different communication styles, it may be desired to obtain a corpus of training data that includes examples of a particular type or style of communication. For example, where a conversation simulator is being used train customer service agents to speak with customers, a training corpus may include examples of conversations between agents and customers).
Regarding claim 17, it recites similar limitations as claim 7 and therefore is rejected similarly.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Farhan in view of Fox and Wang, as applied to claims 1, 6, 9-11, 16, and 19 above, and further in view of Clinchant et al. (US 20230084333 A1; hereinafter referred to as Clinchant).
Regarding claim 8, the combination Farhan, Fox, and Wang teaches: the method of claim 6. The combination Farhan, Fox, and Wang does not explicitly, but Clinchant discloses: wherein the second training text is generated by replacing the first word with the second word not mapped to have a meaning similar to a meaning of the first word ([0106] In example training methods, the adversarial generator aims to learn the masked language mode! (MLM) task, but also to generate data that would break the translation loss. In other words, a combined objective for the example adversarial generator is to produce substitutions that are very likely according to a language model (language model objective) yet would yield a poor translation (adversarial objective)), based on the corpus, and wherein the Al model is trained with the second training text so that a loss between the first training text and third training text output from the Al model in response to application of the second training text to the Al model increases ([0087] example embodiments herein directly learn (train) an adversarial generator. Thus, as opposed to selecting substitute tokens (e.g., words or subwords) a posteriori using the gradient information, example methods disclosed herein provide an adversarial generator that can be directly trained using backpropagation with the goal of maximizing translation loss while also (e.g., during the same forward passes) training the language model for which the adversarial generator provides adversarial examples to improve robustness).
Farhan, Fox, Wang, and Clinchant are considered analogous in the field of language processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan, Fox, and Wang to combine the teachings of Clinchant because doing so would improve the robustness of the language model and reduce training costs by using a loss function (Clinchant [0031] Example methods can improve robustness for NLP models such as NMT models, while avoiding additional search costs (e.g., costs of randomly selecting words and searching for meaning -preserving replacements, which is computationally expensive)).
Regarding claim 18, it recites similar limitations as claim 8 and therefore is rejected similarly.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Farhan in view of Fox and Wang, as applied to claims 1, 6, 9-11, 16, and 19 above, and further in view of Liu et al. (US 20230065965 A1; hereinafter referred to as Liu).
Regarding claim 20, the combination of Farhan, Fox, and Wang teaches: the electronic device of claim 11. The combination of Farhan, Fox, and Wang does not explicitly, but Liu teaches: wherein the at least one NPU ([0147] FIG. 7 is a schematic diagram of a hardware structure of a chip according to an embodiment of this application. The chip includes a neural processing unit (NPU) 40) is configured to perform a preprocessing operation of converting data from the first text applied to the Al model into a form suitable for application to the Al model ([0116] perform related preprocessing on a to-be-processed text (processing may be performed by using a preprocessing module 213 and/or a preprocessing module 214), and then the to -be- processed text is input to the target model/rule 201 for processing, and a processing result corresponding to a target task executed by the target processing model may be obtained).
Farhan, Fox, Wang, and Liu are considered analogous in the field of text processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Farhan, Fox, and Wang to combine the teachings of Liu because doing so would reduce text errors and improve text accuracy by using a corpus-trained language model and a NPU specialized for machine learning (Liu [0298] In this embodiment of this application, a plurality of types of characters in the to -be- processed text can be separately detected and processed. In this way, interference caused by the plurality of types of characters to an error correction process is reduced, accuracy of text error correction is improved, and robustness of the error correction method for the input text is improved).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nathan Tengbumroong whose telephone number is (703)756-1725. The examiner can normally be reached Monday - Friday, 11:30 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN TENGBUMROONG/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights