Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 23 May 2025 and 10 January 2025 are being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 11, 12, 13, 14, and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 11 recites the limitation “the text suggest comprises at least one of: a full form of the following word and a correct orthographic form of the following word”, which is interpreted as the text suggest comprising at least one full form of the following word and at least one correct orthographic form of the following word. However, the applicant’s specification appears to describe that one or both forms of the suggestion may be included, but multiple instances of each form are not included. The examiner recommends 'at least one of' a full form of the following word or a correct orthographic form of the following word, in order to describe that one or both forms of the suggestion may be included.
Claims 12 and 13 are rejected due to their dependence on claim 11.
Claim 14 recites the method of claim 11, further comprising “wherein the outputting comprises outputting the at least one of the full and correct orthographic forms in a descending order of respective values of the ranking parameter thereof”. It should recite ‘one of’ full or correct orthographic forms.
Claim 15 recites the method of claim 14, wherein the ranking parameter is indicative of one of: a position of the text suggest in an alphabetic order; and a confidence level of generating the text suggest. It should recite ‘one of ‘ a position of the text suggest in an alphabetic order or a confidence level of generating the text suggest’.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim [1, 5, 6, 7, 17, 19 ] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.).
Regarding claim 1,
Medlock teaches
A computer-implemented method for generating text suggests for texts
input in one of a plurality of applications executed on an electronic device, the method comprising: receiving, from a user of the electronic device, a textual user input;
[Column 7, lines 25-29 “There is also provided, in accordance with an embodiment, a computer program product including a computer readable medium having stored thereon computer program means for causing a processor to carry out embodiments of the method described herein.”];
[Column 6, lines 33 -41 “In accordance with an embodiment, there is also provided a method for processing user text input and generating text predictions for user selection. The method includes the steps of receiving text input into a user interface, generating concurrently, using a text prediction engine comprising a plurality of language models, text predictions from the multiple language models, and providing text predictions to the user interface for user selection.”];
the NLPM having been trained to generate text suggests based on current user inputs to each one of the plurality of applications based at least in part on the application names thereof;
[Column 4, lines 47- 61 “Preferably, the text predictions are generated concurrently from the plurality of language models in real time. Preferably, the plurality of language models comprises a model of human language and at least one language model specific to an application. More preferably, the at least one language model specific to an application comprises one or more of an email, SMS text, newswire, academic, blog, or product review specific language model. Alternatively, the at least one language model specific to an application comprises an email and an SMS text specific language model and the text predictions are generated using one or both of the email and SMS text specific language models. The plurality of language models may also include at least one additional language model, which may be a user community specific language model for example.”];
[Column 7, lines 58-65 “By way of example only, if the system is a computer or similar device in which the target application is email, then the application specific language model 4 will be a model generated from email language text 2 comprising a large quantity of email messages from a wide variety of authors. Similarly, in the case of a mobile device, the application specific language model 4 will be generated from mobile SMS text language 2.”]
[Column 4, lines 12-15 “FIG. 10 is a block diagram of a text prediction architecture comprising a plurality of trained models used to make probabilistic inferences from a plurality of evidence sources, and a probability generator…;
[Column 23, lines 40-42 “The model set 906 comprises a plurality of trained models representing the plurality of targets (which may be characters) of the system”];
[Column 51, lines 22-34 “The target model set of the target modeling module 82 consists of at least one target model. The target model set can thus have a model for each character in the character pane. Alternatively, each screen mode of the interface 60 (such as all capital letters, lowercase letters, numbers and punctuation, etc.) can have a single model for the corresponding screen mode. In yet another embodiment, a plurality of target models can be maintained for different applications, allowing for a first target model to be trained for a first application, a second target model to be trained for a second application, etc. The target model can be selected automatically based on the application being used, the screen size, the character layout, context information, etc.”];
[Column 29, lines 30-35 ”Thus, the present embodiment provides a system which models user input events for a plurality of targets and updates those models with user input, to provide a system which is able to more accurately predict which character the user intends to input given an input event and thus provide more accurate text predictions.”];
and
outputting the text suggest to enable the user of the electronic device to input the text suggest after the textual user input to the given application.
[Column 4, lines 38-46” a system comprising a user interface configured to receive text input by a user, a text prediction engine comprising a plurality of language models and configured to receive the input text from the user interface and to generate concurrently text predictions using the plurality of language models and wherein the text prediction engine is further configured to provide text predictions to the user interface for display and user selection.”];
However, Medlock does not teach
generating a first vector embedding representative of the textual user input;
feeding the combined vector embedding to a natural language processing model (NLPM) to generate a text suggest for the user to select as input, to the given application, following the textual user input,
But Ball teaches generating a first vector embedding representative of the textual user input;
generating a first vector embedding representative of the textual user input;
[Column 10, lines 54-56 "In FIG. 10, the input text may be processed to compute an input text feature vector that represents the meaning of the input text."];
[Column 10, lines 64-66 "The input text feature vector may be computed by processing the input text with text encoding component 810 to obtain representations for the words of the input text."].
feeding the combined vector embedding to a natural language processing model (NLPM) to generate a text suggest for the user to select as input, to the given application, following the textual user input,
[Column 10, lines 28-33 "Context scoring neural network 920 then processes one or more of the conversation feature vector, the designated message feature vector, the user embedding, and the conversation metadata. For example, the inputs may be concatenated into a single vector and then processed by context scoring neural network 920…." where multiple vectors are combined to single vector];
[Column 10, lines 37-43 "Context scoring neural network 920 then outputs a context score for the designated message that is tailored to the preferences of the user via the user
embedding." where neural network processes the combined vector to determine text suggestion.]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball by representing the textual user input as a feature vector, thereby enabling Medlock’s prediction model to process the semantic meaning of the input efficiently and generate more relevant text suggestions.
Medlock in view of Ball do not teach generating a second vector embedding representative of an application name of a given application of the plurality of applications, to which the textual user input has been made;
However, Li teaches generating a second vector embedding representative of an application name of a given application of the plurality of applications, to which the textual user input has been made;
[0044 " The categorical features in the input to the machine learning model may include, e.g., an application identifier categorical feature, an application developer categorical feature, and an application title categorical feature…
The possible feature values of the application identifier categorical feature may include a predefined set of possible application identifiers ( e.g., represented as integer values), where each application identifier corresponds to a respective application."];
[0051 "The embedding system is configured maintain, for each categorical feature, a respective embedding corresponding to each feature value in a subset of the possible feature values of the categorical feature which are referred to as "active" feature values for the categorical feature…. The embedding system maps each active categorical feature value included in the input 104 to its corresponding embedding, and provides the embeddings to the prediction
system."];
[0052 "The embedding system may map any categorical feature value included in the input that is "inactive", i.e., that is not an active feature value for the categorical feature, i.e.,
that is outside the vocabulary of the categorical feature, to a default (i.e., predefined) embedding, e.g., an embedding including only zeros."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball with Li by representing the application name as a vector embedding, thereby allowing Medlock’s prediction model to distinguish among applications and generate more accurate application specific text suggestions.
Medlock in view of Ball in view of Li do not teach combining the first and second vector embeddings to generate a combined vector embedding for the textual user input;
However, Farooq teaches combining the first and second vector embeddings to generate a combined vector embedding for the textual user input
[Page 4, Column 2 , C. Input Encoding layers “To encode an input review X = (x1, x2, ..., xn), the review encoder first maps each word xi to a high dimensional vector space (i.e., word embedding) embx i , then an RNN is utilized to produce a new d-dimensional representation HX = (hx 1,hx 2,...,hx n) ∈ Rd× n of all the tokens in the input review, where LSTM is used as an RNN.”];
[Page 4, Column 2 Snippets Encoder “The relevant snippets are retrieved by our IR module based on the input review and app name, and then passed to the snippets encoder, which produces a new representation HR = (hr 1,hr 2,...,hr u) ∈ Rd× u where u is the total number of tokens in all the retrieved snippets.”];
[Page 4, Column 2 Category and Rating Encoders. “The category encoder produces a representation hc 1,hc 2,...,hc b for the category of the app and the rating encoder encodes the review rating into hg 1 ∈ Rd. The final hidden states of these layers are passed to the sequence decoder.”];
[Page 4, Column 2 D. Relevant Snippets Fusion layer: This layer associates and fuses information from the relevant snippets and the words of the input review. First, we compute a similarity matrix S ∈ Ru×n between the encodings of the snippets HR and the encodings of the review HX, where Sbk (value at row b and column k) represents the similarity between the b-th word in the snippets and k-th word in the user review, which is computed using Sbk = α(HR :b, HX :k) ∈ R. α is a function trained to capture the similarity between input vectors HR :b and HX :k, where HR :b and HX :k are b th and k-th column-vectors of HR and HX, respectively. α(r, x) = w(s)[r ⊕x ⊕r ⊗u], where ⊕ is vector concatenation, ⊗ is element-wise multiplication, and w(s) is a trainable weight vector. Then, from the similarity matrix S, we can get the most important snippet words with respect to the review, i.e., with the closest similarity to the user review” where HR (snippet) and HK (review) are combined to create a new vector and this new vector is inside the similarity function a(r,x)].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view of Ball in view of Li with Farooq by combining Ball’s textual input vector with Li’s application name vector, as suggested by Farooq’s fusion of text and app-related vectors, thereby enabling the prediction model to produce more relevant application specific suggestions.
Regarding claim 5, the rejection of claim 1 is incorporated.
Medlock in view Ball do not teach the method of claim 1, wherein the combining
comprises summing the first and second vector embeddings.
However, Li teaches the method of claim 1, wherein the combining comprises
summing the first and second vector embeddings.
[0053 "The prediction system is configured to process the embeddings of the categorical feature values included in the input 104 in accordance with values of a set of prediction
system parameters to generate the output 106. For example, to generate the output 106, the prediction system may determine a combined embedding corresponding to each
categorical feature by combining (e.g., summing or averaging) the respective embeddings for each active feature value of the categorical feature that is included in the input 104. The prediction system may then process the combined embeddings corresponding to the categorical features to generate the output 106, e.g., by concatenating the combined embeddings corresponding to the categorical features and
processing the result of the concatenation using one or more neural network layers. The neural network layers may be, e.g., fully-connected neural network layers, convolutional
neural network layers, or any other appropriate type of neural network layers” where two vectors are summed].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball with Li by summing Ball’s textual input vector and Li’s application vector, as taught by Li, to efficiently form a combined representation for generating more accurate application specific text suggestions.
Regarding claim 6, the rejection of claim 1 is incorporated.
Medlock does not teach the method of claim 1, wherein the NLPM comprises a
recurrent neural network (RNN)
But Ball teaches the method of claim 1, wherein the NLPM comprises a
recurrent neural network (RNN).
[Column 9, lines 10- 14 "Response encoder component 820 may process the text encodings using any appropriate neural network, such as a recurrent neural network ( or a bidirectional recurrent neural network or a neural network with a long short-term memory component)."];
[Column 10, lines 9- 18 "For example, context encoder 910 may use one or more neural network layers, such as a recurrent neural network layer (RNN), an RNN with long short-term memory, an RNN with a gated recurrent unit, an RRN with a simple recurrent unit which is incorporated herein by reference in the entirety), a bidirectional RNN, structured self-attention layer, or any neural network layer described herein or in any of the documents incorporated by reference"];
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball by using Ball’s recurrent neural network to process the textual input vector, thereby capturing sequential context and improving the relevance of Medlock’s text suggestions.
Regarding claim 7, the rejection of claim 1 is incorporated.
Medlock does not teach method of claim 1, wherein the NLPM comprises a Long Short-
Term Memory (LSTM) neural network.
But Ball does teach the method of claim 1, wherein the NLPM comprises a Long Short-
Term Memory (LSTM) neural network.
[Column 9, lines 10- 14 "Response encoder component 820 may process the text encodings using any appropriate neural network, such as a recurrent neural network ( or a bidirectional recurrent neural network or a neural network with a long short-term memory component)."];
[Column 10, lines 9- 18 "For example, context encoder 910 may use one or more neural network layers, such as a recurrent neural network layer (RNN), an RNN with long short-term memory, an RNN with a gated recurrent unit, an RRN with a simple recurrent unit which is incorporated herein by reference in the entirety), a bidirectional RNN, structured self-attention layer, or any neural network layer described
herein or in any of the documents incorporated by reference"];
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball by implementing the recurrent neural network as Ball’s long short-term memory network, thereby better preserving preceding textual context and improving Medlock’s text suggestions.
Regarding claim 17, the rejection of claim 1 is incorporated.
Medlock teaches the method of claim 1, wherein the method is executed on the
electronic device.
[Column 42, lines 24-25 “Turning to FIG. 10 and the above described system, the method comprises receiving text input into a user interface, e.g. of an electronic device;”].
Regarding claim 19, the rejection of claim 1 is incorporated.
Medlock does teach an electronic device for generating text suggests for texts
input in one of a plurality of applications executed on the electronic device, the electronic device comprising at least one processor and at
least one non-transitory computer-readable memory storing executable instructions, which, when executed by the at least one processor cause the electronic device to:
receive, from a user of the electronic device, a textual user input;
[Column 7, lines 25-29 “There is also provided, in accordance with an embodiment, a computer program product including a computer readable medium having stored thereon computer program means for causing a processor to carry out embodiments of the method described herein.”];
[Column 6, lines 33 -41 “In accordance with an embodiment, there is also provided a method for processing user text input and generating text predictions for user selection. The method includes the steps of receiving text input into a user interface, generating concurrently, using a text prediction engine comprising a plurality of language models, text predictions from the multiple language models, and providing text predictions to the user interface for user selection.”];
the NLPM having been trained to generate text suggests based on current user inputs to each one of the plurality of applications based at least in part on the application names thereof
[Column 4, lines 47- 61 “Preferably, the text predictions are generated concurrently from the plurality of language models in real time. Preferably, the plurality of language models comprises a model of human language and at least one language model specific to an application. More preferably, the at least one language model specific to an application comprises one or more of an email, SMS text, newswire, academic, blog, or product review specific language model. Alternatively, the at least one language model specific to an application comprises an email and an SMS text specific language model and the text predictions are generated using one or both of the email and SMS text specific language models. The plurality of language models may also include at least one additional language model, which may be a user community specific language model for example.”];
[Column 7, lines 58-65 “By way of example only, if the system is a computer or similar device in which the target application is email, then the application specific language model 4 will be a model generated from email language text 2 comprising a large quantity of email messages from a wide variety of authors. Similarly, in the case of a mobile device, the application specific language model 4 will be generated from mobile SMS text language 2.”]
[Column 4, lines 12-15 “FIG. 10 is a block diagram of a text prediction architecture comprising a plurality of trained models used to make probabilistic inferences from a plurality of evidence sources, and a probability generator…;
[Column 23, lines 40-42 “The model set 906 comprises a plurality of trained models representing the plurality of targets (which may be characters) of the system”];
[Column 51, lines 22-34 “The target model set of the target modeling module 82 consists of at least one target model. The target model set can thus have a model for each character in the character pane. Alternatively, each screen mode of the interface 60 (such as all capital letters, lowercase letters, numbers and punctuation, etc.) can have a single model for the corresponding screen mode. In yet another embodiment, a plurality of target models can be maintained for different applications, allowing for a first target model to be trained for a first application, a second target model to be trained for a second application, etc. The target model can be selected automatically based on the application being used, the screen size, the character layout, context information, etc.”];
[Column 29, lines 30-35 ”Thus, the present embodiment provides a system which models user input events for a plurality of targets and updates those models with user input, to provide a system which is able to more accurately predict which character the user intends to input given an input event and thus provide more accurate text predictions.”];
and
outputting the text suggest to enable the user of the electronic device to input the text suggest after the textual user input to the given application.
[Column 4, lines 38-46 ” a system comprising a user interface configured to receive text input by a user, a text prediction engine comprising a plurality of language models and configured to receive the input text from the user interface and to generate concurrently text predictions using the plurality of language models and wherein the text prediction engine is further configured to provide text predictions to the user interface for display and user selection..”];
However, Medlock does not teach
generating a first vector embedding representative of the textual user input;
feeding the combined vector embedding to a natural language processing model (NLPM) to generate a text suggest for the user to select as input, to the given application, following the textual user input,
But Ball teaches generating a first vector embedding representative of the textual user input;
generating a first vector embedding representative of the textual user input;
[Column 10, lines 54-56 "In FIG. 10, the input text may be processed to compute an input text feature vector that represents the meaning of the input text."];
[Column 10, lines 64-66 "The input text feature vector may be computed by processing the input text with text encoding component 810 to obtain representations for the words of the input text."].
feeding the combined vector embedding to a natural language processing model (NLPM) to generate a text suggest for the user to select as input, to the given application, following the textual user input,
[Column 10, lines 28-33 "Context scoring neural network 920 then processes one or more of the conversation feature vector, the designated message feature vector, the user embedding, and the conversation metadata. For example, the inputs may be concatenated into a single vector and then processed by context scoring neural network 920…." where multiple vectors are combined to single vector];
[Column 10, lines 37-43 "Context scoring neural network 920 then outputs a context score for the designated message that is tailored to the preferences of the user via the user
embedding." where neural network processes the combined vector to determine text suggestion.]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball by representing the textual user input as a feature vector, thereby enabling Medlock’s prediction model to process the semantic meaning of the input efficiently and generate more relevant text suggestions.
Medlock in view of Ball do not teach generating a second vector embedding representative of an application name of a given application of the plurality of applications, to which the textual user input has been made;
However, Li teaches generating a second vector embedding representative of an application name of a given application of the plurality of applications, to which the textual user input has been made;
[0044 " The categorical features in the input to the machine learning model may include, e.g., an application identifier categorical feature, an application developer categorical feature, and an application title categorical feature…
The possible feature values of the application identifier categorical feature may include a predefined set of possible application identifiers ( e.g., represented as integer values), where each application identifier corresponds to a respective application."];
[0051 "The embedding system is configured maintain, for each categorical feature, a respective embedding corresponding to each feature value in a subset of the possible feature values of the categorical feature which are referred to as "active" feature values for the categorical feature…. The embedding system maps each active categorical feature value included in the input 104 to its corresponding embedding, and provides the embeddings to the prediction
system."];
[0052 "The embedding system may map any categorical feature value included in the input that is "inactive", i.e., that is not an active feature value for the categorical feature, i.e.,
that is outside the vocabulary of the categorical feature, to a default (i.e., predefined) embedding, e.g., an embedding including only zeros."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock with Ball with Li by representing the application name as a vector embedding, thereby allowing Medlock’s prediction model to distinguish among applications and generate more accurate application specific text suggestions.
Medlock in view of Ball in view of Li do not teach combining the first and second vector embeddings to generate a combined vector embedding for the textual user input;
However, Farooq teaches combining the first and second vector embeddings to generate a combined vector embedding for the textual user input.
[Page 4, Column 2 , C. Input Encoding layers “To encode an input review X = (x1, x2, ..., xn), the review encoder first maps each word xi to a high dimensional vector space (i.e., word embedding) embx i , then an RNN is utilized to produce a new d-dimensional representation HX = (hx 1,hx 2,...,hx n) ∈ Rd× n of all the tokens in the input review, where LSTM is used as an RNN.”];
[Page 4, Column 2 Snippets Encoder “The relevant snippets are retrieved by our IR module based on the input review and app name, and then passed to the snippets encoder, which produces a new representation HR = (hr 1,hr 2,...,hr u) ∈ Rd× u where u is the total number of tokens in all the retrieved snippets.”];
[Page 4, Column 2 Category and Rating Encoders. “The category encoder produces a representation hc 1,hc 2,...,hc b for the category of the app and the rating encoder encodes the review rating into hg 1 ∈ Rd. The final hidden states of these layers are passed to the sequence decoder.”];
[Page 4, Column 2 D. Relevant Snippets Fusion layer: This layer associates and fuses information from the relevant snippets and the words of the input review. First, we compute a similarity matrix S ∈ Ru×n between the encodings of the snippets HR and the encodings of the review HX, where Sbk (value at row b and column k) represents the similarity between the b-th word in the snippets and k-th word in the user review, which is computed using Sbk = α(HR :b, HX :k) ∈ R. α is a function trained to capture the similarity between input vectors HR :b and HX :k, where HR :b and HX :k are b th and k-th column-vectors of HR and HX, respectively. α(r, x) = w(s)[r ⊕x ⊕r ⊗u], where ⊕ is vector concatenation, ⊗ is element-wise multiplication, and w(s) is a trainable weight vector. Then, from the similarity matrix S, we can get the most important snippet words with respect to the review, i.e., with the closest similarity to the user review” where HR (snippet) and HK (review) are combined to create a new vector and this new vector is inside the similarity function a(r,x)].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view of Ball in view of Li with Farooq by combining Ball’s textual input vector with Li’s application name vector, as suggested by Farooq’s fusion of text and app-related vectors, thereby enabling the prediction model to produce more relevant application specific suggestions.
Claim [ 2 ] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Walters (US1071966, hereinafter Walters).
Regarding claim 2, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of
claim 1, wherein the generating the first vector embedding comprising using a text embedding algorithm based on a convolutional neural network (CNN).
However, Walter teaches the method of claim 1, wherein the generating the first
vector embedding comprising using a text embedding algorithm based on a convolutional neural network (CNN).
[Column 5, lines 21-24 " The machine learning algorithm may map the candidate phrase into a candidate embedding vector in the n-dimensional textual embedding space."];
[Column 5, lines 11-17 "The plurality of phrases may be input into a machine learning algorithm, such as trained neural network model, that may be configured to map each of the words and/or phrases of words in the plurality of phrases and/or the discourse of the plurality of phrases into a respective plurality of embedding vectors in an n-dimensional textual embedding space."];
[Column 9, lines 18-22 "In some embodiments, the machine learning model may include, for example, a neural network model such as a bidirectional encoder representations from transformers (BERT) model, a convolutional neural network model, and/or a recurrent neural network model."]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Walter by generating the textual input vector using Walter’s CNN based embedding algorithm, thereby efficiently capturing local textual patterns and improving the accuracy of application specific text suggestions.
Claim [3, 20] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) in further view of Walters (US10719666, hereinafter Walters) and in further view of Sankaran (US 11301640 B2, hereinafter Sankaran).
Regarding claim 3, the rejection of claim 2 is incorporated.
Medlock in view Ball in view of Li in view of Farooq in view of Walter do not teach the method of claim 2, wherein the text embedding algorithm is a CHAR-CNN embedding algorithm.
However, Sankaran teaches the method of claim 2, wherein the text embedding
algorithm is a CHAR-CNN embedding algorithm.
[Column 6, lines 7- 10 "Referring to FIG. 3, latent representation of the input text can be obtained using text encoders such as a character level convolutional neural network (CNN) or long-short term memory (LSTM). "];
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq in view Walter with Sankaran by using Sankaran’s character-level CNN to generate the textual input embedding, thereby capturing character patterns and misspellings and improving application specific text suggestions.
Regarding claim 20, the rejection of claim 2 is incorporated. Claim 20 is substantially the same as claim 3 and is therefore rejected under the same rationale as above.
Claim [4] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) in further view of Gupta (US 11977660 B2, hereinafter Gupta).
Regarding claim 4, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of
claim 1, wherein the generating the second vector embedding comprises applying a one-hot encoding algorithm.
However, Gupta teaches the method of claim 1, wherein the generating the
second vector embedding comprises applying a one-hot encoding algorithm.
[Column 11, lines 9-14 "The analytics server may use one or more of the following methods of text encoding: "One-Hot Encoding—One-Hot Encoding protocol may create additional features based on the number of unique values in the categorical feature."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Gupta by using Gupta’s one-hot encoding algorithm to encode the application name, thereby providing a simple and distinct numerical representation of each application for generating application specific text suggestions.
Claim [8] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.). and in further view of Peng (Peng, Bo, et al. "Rwkv: Reinventing rnns for the transformer era." Findings of the association for computational linguistics: EMNLP 2023. 2023., hereinafter Peng)
Regarding claim 8, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of
claim 1, wherein the NLPM comprises Receptance Weighted Key Value (RWKV) neural network.
However, Peng teaches the method of claim 1, wherein the NLPM comprises a
Receptance Weighted Key Value (RWKV) neural network.
[Abstract "In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and computational requirements but struggle to match the same performance as Transformers due to limitations in parallelization and scalability. We propose a novel model architecture, Receptance Weighted Key Value (RWKV), that combines the efficient parallelizable training of transformers with the efficient inference of RNNs."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Peng by implementing the NLP model as Peng’s RWKV neural network, thereby efficiently capturing long-range textual dependencies while improving the generation of application specific text suggestions.
Claim [9] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Griffin (US 9,134,810 B2, hereinafter Griffin).
Regarding claim 9, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do no teach the method of claim 1, wherein: the textual user input has been made by swiping over a virtual keyboard of the electronic device with an intent to input a given symbol of a given word; and the text suggest comprises a symbol in the given word following immediately after the given symbol.
However, Griffin teaches the method of claim 1, wherein: the textual user input
has been made by swiping over a virtual keyboard of the electronic device with an intent to input a given symbol of a given word; and the text suggest comprises a symbol in the given word following immediately after the given symbol.
[Column 9, 10, lines 65-67 ,1-4 "A user could input a generated set of characters in various ways, including in a way that differs from a manner of inputting a character key. For example, to
input a generated set of characters, a user could use a finger or stylus to swipe the generated set of characters. As used herein, swiping includes swiping the set of characters itself or swiping or touching near the set of characters"];
[Column 9, lines 26-36 " In the example shown in FIG. 3B, “P” is received as input and a predictor generates several set of characters 360, which are displayed at keys corresponding to each generated set of characters' subsequent candidate input character. As shown in FIG. 3B, “People” is placed at the “E” key because the next letter after “P” of “People” is “E”; “Paul” will be place at the “A” key because the next letter after “P” of “Paul” is “A”; “Phone” will be placed at the “H” key because the next letter after “P” of “Phone” is “H”; and so on. It should be noted that any of the letters in the set of characters can be upper case or lower case"].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Griffin because by using swipe input over a virtual keyboard to identify an intended symbol and suggest the immediately following symbol of the intended word, thereby enabling faster and more accurate text entry.
Claim [10] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Griffin (US 9,134,810 B2, hereinafter Griffin).and in further view of Kataoka(US-9304595-B2, hereinafter Kataoka).
Regarding claim 10, the rejection of claim 9 is incorporated.
Medlock in view Ball in view of Li in view of Farooq in view of Griffin do not teach
the method of claim 9, further comprising determining the intent based on a curve defined by the swiping over the virtual keyboard.
However, Kataoka teaches the method of claim 9, further comprising determining the intent based on a curve defined by the swiping over the virtual keyboard.
[Column 3, lines 21-26 "In certain examples, the search may include determining one or more selected keys of a graphical keyboard based at least in part on a plurality of features of the received gesture input, such as a speed of one or more segments of the gesture, a direction of a segment of the gesture, a curvature of a segment of the gesture, etc."];
[Column 5, lines 30-36 "In some examples, gesture module 8 can determine one or more features associated with a gesture, such as the Euclidean distance between two alignment points, the length of a gesture path, the direction of a gesture, the curvature of a gesture path, the shape of the gesture, and maximum curvature of a gesture between alignment points, speed of the gesture, etc."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view in view of Farooq in view of Griffin with Kataoka by determining the user’s intended input from the swipe curve over the virtual keyboard, thereby improving the accuracy of swipe based text suggestions.
Claim [11, 13, 16 ] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Anand (US-11748578-B1, hereinafter Anand).
Regarding claim 11, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of
claim 1, wherein:
the textual user input comprises a given word and a prefix of a following word; and
the text suggest comprises at least one of: a full form of the following word and a correct orthographic form of the following word.
However, Anand teach the method of claim 1, wherein:
the textual user input comprises a given word and a prefix of a following word; and
the text suggest comprises at least one of: a full form of the following word and a correct orthographic form of the following word.
[Column 2, lines 28-34 "Predictive text systems may take into consideration the previous word entered in presenting suggestions. For example, if a user types “I'll” suggestions may appear—before the user even enters in another character—such as “be” or “do” or “have.” The suggestions may be refined as a user type. Thus, if a user types ‘d’ the suggestions may be updated to “do” and “definitively.”"];
[Column 6, 7, lines 67, 1-4 "Suggestion component 126 may submit input the characters, in vector form, into neural network 130 at predictive text system 106. As discussed above, the output of neural network 130 may indicate the most likely completed or next term"];
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Anand by suggesting a full form or corrected orthographic form of the following word, thereby improving the accuracy and efficiency of text entry.
Regarding claim 13, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of claim 11, wherein the correct orthographic form of the following word comprises a word combination including the following word.
However, Anand teaches the method of claim 11, wherein the correct orthographic
form of the following word comprises a word combination including the following word.
[Column 10, lines 23- 28 "As illustrated, a user may have entered “Bi” into input box 502. Input box 502 may be an example of text input interface 108. Input processing 110 may use “Bi” as an input into neural network 130. In response, a set of suggestions may be generated as discussed previously which include “Bill Pay”, “Billing Cycle” and “Billing Date”."]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Anand by providing the corrected orthographic form as a word combination including the following word, thereby producing more contextually accurate and complete text suggestions.
Regarding claim 16, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq do not teach the method of
claim 1, wherein the text suggest for the textual user input in the given application is different from another text suggest for the textual user input in another application of the plurality of application of the electronic device.
However, Anand teaches the method of claim 1, wherein the text suggest for
the textual user input in the given application is different from another text suggest for the textual user input in another application of the plurality of application of the electronic device.
[Column 2, lines 49-60 " In view of the above problems and limitations of existing predictive text systems, an improved predictive text mechanism may be used, The predictive text system may be used within a financial application installed on a user's mobile device, such as a cellular phone in various examples. The predictive text system may use a trained machine learning model, such as a neural network, to suggest words or phrases (herein referred as terms) based on characters input into a keyboard. Although discussed in a financial context, the described solution is not limited to such uses. For example, other specialized or context specific domains (e.g., medical) may train their own neural networks to suggest terms to a user.."];
[Column 3, lines 18-20 "For example, if the user has a mortgage account with a financial institution, and types “mort” a suggested term may be “mortgage balance: $234,124.”];
[Column 4 , lines 48-59 "Mobile application 104 may be used to process input in a chat application. The chat application may be associated with the same organization/enterprise as mobile application
104 or be a third-party application (e.g., Messenger by the FACEBOOK® social network). If a third-party application is used, mobile application 104 may restrict some of the suggestions regardless of the authentication status of the user. Restrictions may include preventing suggestions with account numbers or balances. Accordingly, in some instances, the predictive keyboard may function as financial specific predictive keyboard in contrast with a financial specific predictive keyboard customized to a user."];
[Column 4, 5 , lines 66-68, lines 1-3 "As with third-party applications, suggestions may be restricted when using mobile application 104 as a replacement keyboard outside of an application associated with the organization providing the keyboard."]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq with Anand by generating different text suggestions for the same textual input in different applications, thereby tailoring the suggestions to each application’s context.
Claim [12, 14] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Anand (US-11748578-B1, hereinafter Anand) and in further view of Gupta (US 10,936,813 B 1, hereinafter Gupta1).
Regarding claim 12, the rejection of claim 1 is incorporated.
Medlock in view Ball in view of Li in view of Farooq in view of Anand do not teach
the method of claim 11, wherein the full form of the following word includes a list of full form candidates for the following word.
However, Gupta1 teaches the method of claim 11, wherein the
full form of the following word includes a list of full form candidates for the following word.
[Column 9, lines 4-7 "To illustrate, and utilizing the example shown in FIG. 1, candidate suggestions for the token “dg” may include “dog,” “dig,” and “dug.”"];
[Column 3, 4 lines 64-67, lines 1-8 "For example, for the candidate suggestions, the context-aware spell checker may determine the frequency of occurrence for the trigrams “he drinks coffee” and “he drinks coffer.” In other words, using the frequencies of the unigrams, bigrams, and/or trigrams stored in the corpus, the context-aware spell checker may determine a score, or likelihood, that the candidate suggestion pertains to an intended word of the non-word spelling error. Accordingly, using the n-gram conditional probabilities, the context-aware spell checker may utilize words surrounding the non-word spelling error to determine context and suggest corrections."];
[Column 2, 3 lines 60-67, 1-8 "On a word level, in the sentence “he drinks coffee,” the unigrams are “he,” “drinks,” and “coffee”, the bigrams are “he drinks” and “drinks coffee”, and the trigram is “he drinks coffee.” In this sense, a bigram, for example, represents the occurrence of a word based on the preceding word (i.e., n−1) or the succeeding word (i.e., n+1). In generating the corpus, the frequency of occurrence for the n-grams, whether on a character level and/or the word level, may be determined. For example, on a character-level, the corpus may store an indication of how many times the letter “e” follows the letter “h” (e.g., how often are they present in the same word or phrase, at what frequency they follow one another, etc.). On a word-level, the corpus may store an indication of how many times the word “coffee” follows the word “drinks,” for example. "]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq in view of Anand with Gupta1 by providing a list of full-form candidates for the following word, thereby allowing selection of the most appropriate completion and improving text-entry efficiency.
Regarding claim 14, the rejection of claim 11 is incorporated.
Medlock in view Ball in view of Li in view of Farooq in view of Anand do not teach
the method of claim 11, further comprising:
ranking the at least one of the full and the correct orthographic forms of the following word according to a respective value of a ranking parameter thereof; and
wherein the outputting comprises outputting the at least one of the full and correct orthographic forms in a descending order of respective values of the ranking parameter
thereof.
However, Gupta1 teaches the method of claim 11, further comprising:
ranking the at least one of the full and the correct orthographic forms of the following word according to a respective value of a ranking parameter thereof; and
wherein the outputting comprises outputting the at least one of the full and correct orthographic forms in a descending order of respective values of the ranking parameter thereof.
[Column 13, lines 31-34 "Utilizing the n-gram conditional probabilities, the spell checker component 102 may rank the candidate suggestions at 210 based on a cumulative score of the unigrams, bigrams, and/or trigrams"];
[Column 8, lines 25-28 "For each candidate suggestion, a score may be calculated, where the score represents the likelihood of the candidate suggestion corresponding to the likelihood of the candidate suggestion being the intended word"];
[Column 4, lines 22-32 "For each candidate suggestion, this process may be repeated and the context-aware spell checker may present the candidate suggestions with the highest scores. In some instances, the context-aware spell checker may present candidate suggestions having a cumulative score greater than or equal to a threshold amount or the context-aware spell checker may present a certain number of candidate suggestions (e.g., top five, top ten, etc.). Prioritizing the candidate suggestions may accordingly list words in order that are most natural in the context of the text."];
[Column 9, lines 41- 45 "Upon identifying the candidate suggestions, determining the cumulative score for each of the candidate suggestions (i.e., the unigram, bigram, and/or trigram n-conditional probabilities), the candidate suggestions may be presented as a prioritized list. "];
[Column 9, lines 54-63 "In the illustrative example, as the corpus 118 likely contains a higher frequency of n-grams corresponding to “walk my dog,” than “walk my dig” or “walk my dug,” for example, the n-gram conditional probability for the candidate suggestion of “dog” may be higher than “dig” or “dug.” In this sense, using the corpus 118 and the conditional probabilities 136, the n-gram component 114 may determine a probability that the word “dog” follows the bigram “walk my.” Similar n-gram conditional probabilities for unigrams and trigrams may be calculated."].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq in view of Anand with Gupta1 by ranking the full form and corrected orthographic candidates according to respective ranking values and presenting them in descending order, thereby showing the most useful text suggestion first.
Claim [15] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Anand (US-11748578-B1, hereinafter Anand) and in further view of Gupta (US 10,936,813 B 1, hereinafter Gupta1) and in further view of Bailey (US-11573989-B2, hereinafter Bailey).
Regarding claim 15, the rejection of claim 14 is incorporated.
Medlock in view Ball in view of Li in view of Farooq in view of Anand and in view
of Gupta do not teach the method of claim 14, wherein the ranking parameter is indicative of one of: a position of the text suggest in an alphabetic order; and a confidence level of generating the text suggest.
However, Bailey teaches the method of claim 14, wherein the ranking parameter is
indicative of one of: a position of the text suggest in an alphabetic order; and a confidence level of generating the text suggest.
[Column 8, lines 66- 67 "Yet another example may rank the suggestions in alphabetical order. "];
[Column 10, lines 19- 25 "Selection can be made when a score exceeds a threshold. The score can be any of the above measures/metrics, a combination of the above measures/metrics (i.e., combined into a single score as with a weighted sum or weighted average), and/or multiple of the above measures/metrics in any combination (i.e., metric A is above one threshold and metric B is below another threshold, etc.)." where scoring determines the confidence level of suggested text];
[Column 10, line 31- 32 "The score can be any rank, measure, a-priori probability, etc. that indicates a score or ranking of the entry"]
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li in view of Farooq in view of Anand and in view of Gupta with Bailey by ranking text suggestions according to alphabetical position and generation confidence, thereby providing an organized list that priorities the most reliable suggestions.
Claim [18 ] are rejected under 35 U.S.C. 103 as being unpatentable over Medlock (US9189472 B2, hereinafter Medlock) in view of Ball (US11425064B2, hereinafter Ball) in view of Li (US2020/0372076A1, hereinafter Li) and in further view of Farooq (Farooq, Umar, et al. "App-aware response synthesis for user reviews." 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2020.) and in further view of Fusco (US20240289683, hereinafter Fusco).
Regarding claim 18, the rejection of claim 1 is incorporated.
Medlock in view of Ball teaches the following.
Ball further teaches
[Column 10 , lines 54-57 "In FIG. 10, the input text may be processed to compute an input text feature vector that represents the meaning of the input text. An input text feature vector is a vector in a vector space that represents the meaning of the input text."];
[Column 10, lines 64-66 "The input text feature vector may be computed by processing the input text with text encoding component 810 to obtain representations for the words of the input text"];
[Column 11 lines "The text encodings may then be processed by response encoder component 820 to compute the input text feature vector."];
[Column 8, lines 13-15 " A word embedding is a vector in a vector space that represents the word but does so in a manner that preserves useful information about the meaning of the word"];
[Column 8, lines 22-25 "Word embeddings may be trained in advance using a training corpus, and when obtaining the word embeddings, a lookup may be performed to obtain a word embedding for each word of the message."];
a third training vector embedding, representative of another training textual user input to the training application following the given textual user input;
[Column 12, lines 29-38 " For a first message transmitted by the first user, the inputs and outputs of FIG. 9 may be configured as follows: the text of the first conversation prior to the first message may be input as the conversation text; a first message feature vector may by computed from the first message (e.g., computed as described in FIG. 8 for the designated messages) and the first message feature vector may be input as the designated message feature vector;" the text of the first conversation before the first message corresponds to the given training textual user input. The first message subsequently transmitted by the user corresponds to the other training textual input to the training application. The first message feature vector corresponds to the third training vector embedding representative of the other training textual input];
["Column 9, lines 5-8 "Response encoder component 820 may process the text encoding of the designated message and compute a designated message feature vector that represents the designated message in a vector space "]
[Column 9, lines 27-29 "Response encoder component 820 then outputs a designated message feature vector for the input designated message."];
and feeding the given training digital object of the plurality of training digital objects to the NLP,
[Column 10, lines 28-333 "Context scoring neural network 920 then processes one or more of the conversation feature vector, the designated message feature vector, the user embedding, and the conversation metadata.
For example, the inputs may be concatenated into a single vector and then processed by context scoring neural network 920."]
However, Medlock in view of Ball do not teach the method of claim 1, further
comprising training the NLPM by: acquiring a training set of data, the training set of data comprising a plurality of training digital objects, a given one of which includes:
But Li teaches the method of claim 1, further comprising training
the NLPM by: acquiring a training set of data, the training set of data comprising a plurality of training digital objects, a given one of which includes:
[0060 "The training system 100 provides the categorical feature value embeddings (i.e., which conform to the categorical feature specification 112) to the machine learning model 102, and uses a training engine 108 to train the machine learning model 102 on a set of training data 114.];
[0061 "The training engine 108 may train the machine learning model 102 on the training data 114 using multiple iterations of stochastic gradient descent to optimize an objective function, e.g., that measures the prediction accuracy of the machine learning model 102, e.g., a cross-entropy objective function."];
[0060 "…The training data includes a set of training examples, where each training example specifies: (i) a training input to the machine learning model, and (ii) a target output that should be generated by the machine learning model by processing the training input." where each training example corresponds to one claimed training digital object];
(ii) a second training vector embedding representative of a training name application of the training
[0044 "The categorical features in the input to the machine learning model may include, e.g., an application identifier categorical feature, an application developer categorical feature, and an application title categorical feature. The possible feature values of the application identifier categorical feature may include a predefined set of possible application identifiers (e.g., represented as integer values), where each application identifier corresponds to a respective application."];
[0051 "As used throughout this specification, an “embedding” refers to an ordered collection of numerical values, e.g., a vector or matrix of numerical values. The embedding system maps each active categorical feature value included in the input 104 to its corresponding embedding, and provides the embeddings to the prediction system."];
[0059 "The training system 100 then obtains a set of categorical feature value embeddings that conform to the categorical feature specification 112. That is, the training system 100 obtains a respective embedding for each feature value in the vocabulary of each categorical feature,.."];
[0061 "The training engine 108 may train the machine learning model 102 on the training data 114 using multiple iterations of stochastic gradient descent to optimize an objective function, e.g., that measures the prediction accuracy of the machine learning model 102, e.g., a cross-entropy objective function. At each iteration of stochastic gradient descent, the training engine 108 may backpropagate gradients of the objective function through the machine learning model to adjust the embeddings of the active feature values of each categorical feature"];
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball with Li to represent the textual user input and application name as respective vectors embeddings, because doing so would allow Medlock’s prediction model to efficiently incorporate both textual and application specific content, thereby improving the accuracy of application specific text suggestions.
Medlock in view of Ball in view of Li view of Farooq do not teach (iii) a respective label including a third training vector embedding,
However, Fusco teaches (iii) a respective label including a third training vector embedding,
[0063"…this model has been trained on a training dataset 14 associating training terms 145 with first embeddings 146, so as to learn to generate second embeddings"];
[0051" During the training phase, the term encoder learns its own parameters by minimizing distances between the second (i.e., reconstructed) embeddings and the first embeddings 146." where training term correspond to the other training textual input which could be the training textual input from Ball. First embedding 146 (respective label) corresponds to the third training vector embedding included in the respective target label. The generated second embedding corresponds to the current prediction of the NLP model. The distance between the generated second embedding and first target embedding correspond to minimizing the difference between current prediction of the NLP model and the respective label.].
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Medlock in view Ball in view of Li view of Farooq with Fusco by using the embedding of the actual following text as a target label and minimizing its distance from the predicted embedding, thereby training the model to generate more accurate application specific text suggestions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEZA ABDUL AZIZ whose telephone number is (571)272-9610. The examiner can normally be reached Monday-Friday 7:30am-5pm Alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHEZA ABDUL AZIZ/Examiner, Art Unit 2657
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights