DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
Claims 1-20 are non-provisionally rejected on the ground of obviousness-type nonstatutory double patenting as being unpatentable over claims 1-36 US patent 19228287. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the copending application disclose the function and structure of the claims of the instant application to those having ordinary skill in the art.
6/04/2025 – Current application – claim 1
5/20/2022 – 19228287– claim 1
An apparatus comprising: machine-readable instructions; and programmable circuitry to be programmed by the machine-readable instructions to: execute a first machine-learning model with first and second text portions of content as input, the first machine-learning model executed to encode the first text portion to define a first vector and encode the second text portion to define a second vector; determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; encode the third text portion to define a third vector different from the first vector and the second vector; execute a second machine learning model based on a query to obtain related search results corresponding to the third vector; update telemetry data based on the related search results and the query, the telemetry data included in training data for training at least one of the first machine- learning model or the second machine-learning model; and trigger re-training of at least one of the first machine-learning model or the second machine-learning model based on the updated telemetry data
An apparatus comprising: memory to store machine-readable instructions; and at least one processor to execute the machine-readable instructions to at least: tokenize text for from content into text portions, the text portions including a first text portion and a second text portion; execute a first machine-learning model with the text portions as input to the first machine-learning model to encode the first text portion to define as a first vector and encode the second text portion to define a second vector, the first machine-learning model to output the first vector and the second vector, the first machine- learning model trained with contextually similar text data and context corresponding to the contextually similar text data, the context associated with a pattern represented by the contextually similar text data; determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; encode the third text portion to define a third vector different from the first vector and the second vector; cause storage of the third vector to define a vector database; query the vector database to extract related search results from the vector database by comparing the third vector to a query vector corresponding to a query; and execute a second machine-learning model different from the first machine- learning model with the related search results as input to the second machine- learning model to generate rankings of the related search results as output from the second machine-learning model for presentation on a computing device, the second machine-learning model trained with data corresponding to at least one query and search results associated with the at least one query
Corresponding product claim 8 is rejected similarly as claim 1 above
Corresponding method claim 15 is rejected similarly as claim 1 above
6/04/2025 – Current application – claim 2
5/20/2022 – 19228287– claim 2
The apparatus of claim 1, wherein the programmable circuitry is to execute a third machine-learning model based on the related search results to output rankings of the related search results for presentation on a computing device.
The apparatus of claim 1, wherein the at least one processor is to execute the machine-readable instructions to: obtain the query from the computing device via a network; and transmit the related search results to the computing device via the network.
Corresponding product claim 8 is rejected similarly as claim 2 above
Corresponding method claim 15 is rejected similarly as claim 2 above
6/04/2025 – Current application – claim 3
5/20/2022 – 19228287– claim 1
The apparatus of claim 2, wherein the rankings are based on similarities between first ones of the related search results and second ones of the related search results.
An apparatus comprising: memory to store machine-readable instructions; and at least one processor to execute the machine-readable instructions to at least: tokenize text for from content into text portions, the text portions including a first text portion and a second text portion; execute a first machine-learning model with the text portions as input to the first machine-learning model to encode the first text portion to define as a first vector and encode the second text portion to define a second vector, the first machine-learning model to output the first vector and the second vector, the first machine- learning model trained with contextually similar text data and context corresponding to the contextually similar text data, the context associated with a pattern represented by the contextually similar text data; determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; encode the third text portion to define a third vector different from the first vector and the second vector; cause storage of the third vector to define a vector database; query the vector database to extract related search results from the vector database by comparing the third vector to a query vector corresponding to a query; and execute a second machine-learning model different from the first machine- learning model with the related search results as input to the second machine- learning model to generate rankings of the related search results as output from the second machine-learning model for presentation on a computing device, the second machine-learning model trained with data corresponding to at least one query and search results associated with the at least one query
Corresponding product claim 9 is rejected similarly as claim 3 above
Corresponding method claim 16 is rejected similarly as claim 3 above
6/04/2025 – Current application – claim 4
5/20/2022 – 19228287– claim 8
The apparatus of claim 1, wherein the telemetry data includes at least one of frequencies corresponding to processed queries or metadata corresponding to the processed queries.
The apparatus of claim 1, wherein the at least one processor is to execute the machine-readable instructions to: generate an application including a user interface to obtain the query and a telemetry agent to generate telemetry data based on the query; and transmit the application to the computing device via a network, the computing device to execute the application in response to the transmitting.
Corresponding product claim 10 is rejected similarly as claim 4 above
Corresponding method claim 17 is rejected similarly as claim 4 above
6/04/2025 – Current application – claim 5
5/20/2022 – 19228287– claim 1
The apparatus of claim 1, wherein the programmable circuitry is to arrange the first text portion and the second text portion based on the natural language similarity.
An apparatus comprising: memory to store machine-readable instructions; and at least one processor to execute the machine-readable instructions to at least: tokenize text for from content into text portions, the text portions including a first text portion and a second text portion; execute a first machine-learning model with the text portions as input to the first machine-learning model to encode the first text portion to define as a first vector and encode the second text portion to define a second vector, the first machine-learning model to output the first vector and the second vector, the first machine- learning model trained with contextually similar text data and context corresponding to the contextually similar text data, the context associated with a pattern represented by the contextually similar text data; determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; encode the third text portion to define a third vector different from the first vector and the second vector; cause storage of the third vector to define a vector database; query the vector database to extract related search results from the vector database by comparing the third vector to a query vector corresponding to a query; and execute a second machine-learning model different from the first machine- learning model with the related search results as input to the second machine- learning model to generate rankings of the related search results as output from the second machine-learning model for presentation on a computing device, the second machine-learning model trained with data corresponding to at least one query and search results associated with the at least one query
Corresponding product claim 11 is rejected similarly as claim 5 above
Corresponding method claim 18 is rejected similarly as claim 5 above
6/04/2025 – Current application – claim 6
5/20/2022 – 19228287– claim 1
The apparatus of claim 1, wherein the threshold is a first threshold, and the programmable circuitry is to trigger the re-training when a quantity corresponding to the training data satisfies a second threshold.
An apparatus comprising: memory to store machine-readable instructions; and at least one processor to execute the machine-readable instructions to at least: tokenize text for from content into text portions, the text portions including a first text portion and a second text portion; execute a first machine-learning model with the text portions as input to the first machine-learning model to encode the first text portion to define as a first vector and encode the second text portion to define a second vector, the first machine-learning model to output the first vector and the second vector, the first machine- learning model trained with contextually similar text data and context corresponding to the contextually similar text data, the context associated with a pattern represented by the contextually similar text data; determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; encode the third text portion to define a third vector different from the first vector and the second vector; cause storage of the third vector to define a vector database; query the vector database to extract related search results from the vector database by comparing the third vector to a query vector corresponding to a query; and execute a second machine-learning model different from the first machine- learning model with the related search results as input to the second machine- learning model to generate rankings of the related search results as output from the second machine-learning model for presentation on a computing device, the second machine-learning model trained with data corresponding to at least one query and search results associated with the at least one query
Corresponding product claim 13 is rejected similarly as claim 6 above
Corresponding method claim 20 is rejected similarly as claim 6 above
6/04/2025 – Current application – claim 7
5/20/2022 – 19228287– claim 4
The apparatus of claim 1, wherein the programmable circuitry is to determine the natural language similarity by determining a cosine similarity between the first text portion and the second text portion.
The apparatus of claim 1, wherein the natural language similarity is a cosine similarity
Corresponding product claim 14 is rejected similarly as claim 7 above
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20190130073 A1; Sun; Weiyi et al. (hereinafter Sun) in view of US 20210141798 A1; Steedman Henderson; Matthew (hereinafter Steedman).
Regarding claim 1, Sun teaches An apparatus comprising:machine-readable instructions; andprogrammable circuitry to be programmed by the machine-readable instructions to:execute a first machine-learning model with first and second text portions of content as input, the first machine-learning model executed to encode the first text portion to define a first vector and encode the second text portion to define a second vector (Sun [0178] machine learning model ...provide a combination of both rules-based and statistics-based, or otherwise train a DLR component configured to... determine whether a region of text from documentation ... training data 1895 may be introduced to a machine learning model to learn the characteristics of text [0179] FIG. 19 illustrates a DLR component 1965 that comprises a DLR model 1969 that is trained using features 1995 extracted from training data 1895 by feature extractor 1963. For example, feature extractor 1963 may be configured to extract salient features from training data 1895 that facilitate DLR model 1969 learning the characteristics of text regions that are prone to give rise to false positive medical billing code suggestions and/or characteristics of text regions that tend to give rise to true positive medical billing code suggestions. As discussed above, once trained, DLR component 1965 may be used to evaluate the likelihood that text regions from a given text 1410 will result in one or more false positive medical billing code suggestions based on the learned characteristics and/or context. A number of rule based and/or statistic models (e.g., machine learning models such as statistical classifiers [0187] In act 2120, a first text region of the plurality of text regions is applied to the language embedding model for training. According to some embodiments, the language embedding model is a word2vec or paragraph2vec technique in which a word is input into the model and a representative vector is output. The model may be, for example, a neural network or other suitable machine learning framework. According to some embodiments, a text region is applied to the language embedding model by providing each word in the text region as input to the language embedding model. For example, in act 2130, a first word in the first text region may be provided as input to the language embedding model and to produce an output vector. The output vector is fed back to the language embedding model as an input in conjunction with the next word in the first text region. [180-188 & 201] elaborate on the matter of using a ML to executed to encode the first text portion to define a first vector and encode the second text portion to define a second vector [FIG.19 in conjunction with FIG.20B] show execute a first machine-learning model with first and second text portions of content as input, the first machine-learning model executed to encode the first text portion to define a first vector and encode the second text portion to define a second vector ) determine, based on a comparison between the first vector and the second vector, a natural language similarity between the first text portion and the second text portion; (Sun [0099] closest matching terms may be generated, and may be ranked by their similarity to the tokens in the text. The similarity may be scored in any suitable way. For example, in one suitable technique, one or more tokens in the text may be considered as a vector of its component elements, such as words, and each of the terms in the ontology may also be considered as a vector of component elements such as words. Similarity scores between the tokens may then be computed by comparing the corresponding vectors, e.g., by calculating the angle between the vectors, or a related measurement such as the cosine of the angle. In some embodiments, one or more concepts that are linked in the ontology to one or more of the higher ranking terms (e.g., the terms most similar [180] a clustering algorithm can be performed. For example, a text-to-vector representation made be used to convert text regions into vectors that can be compared in vector space. By transforming text regions into a vector space, any of various clustering techniques may be used to identify clusters of training data that are “near” each other in a given vector space [0181] The inventors have recognized that language embedding may be used to derive a representation of text regions ... language embedding techniques may transform text into a vector space where semantically similar text appears closer in vector space than does semantically dissimilar text and/or where text with similar content appears closer in vector space than does text with dissimilar content. In this manner, text regions that are not diagnostically relevant may transform into vectors that tend to cluster in vector space and text regions that do include diagnostically relevant information (e.g., relevant from a billing perspective) may also tend to cluster in vector space.[181-185] goes into details on a natural language similarity between the first text portion and the second text portion [FIG.20B] shows corresponding visual) in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion; (Sun [0070] text documents may be marked up (e.g., by a human) with labels indicating the nature of the relevance of particular statements in the text to the patient encounter or medical topic to which the text relates. A statistical knowledge representation model may then be trained to form associations based on the prevalence of particular labels corresponding to similar text within an aggregate set of multiple marked up documents [184] data 2010 can be segmented into paragraphs (or an approximation of paragraphs) to provide a plurality of text regions with which to train language embedding model 2071. Small text regions (e.g., text regions that have fewer than a threshold number of words) may be appended or prepended to an adjacent text region. It should be appreciated that segmenting training data 2010 into paragraphs is only one method of partitioning the training data and training data 2010 may be parsed into text regions into any desired grouping of words (e.g., sentences) in any suitable way[0185] Thus, features 2067 provided to language embedding model may be words grouped into paragraphs (or an estimate of paragraphs) or words grouped into text regions based on proximity or one or more other factors or criteria (e.g., words in a section, field, etc.). Features 2067 are then provided to language embedding model 2071 to train the language embedding model to produce an output 2075 that may be representative of some aspect of the text region (e.g., semantic content, word content, contextual meaning, etc.). According to some embodiments, the result of training language embedding model 2071 is that the trained model 2071′ produces output 2075 that tends to be more similar for semantically similar text regions and tends to be more dissimilar for semantically dissimilar text regions. According to some embodiments, the result of training language embedding model 2071 is that the trained model 2071′ produces output 2075 that tends to be more similar for text regions that have similar word content and/or word arrangement[0194]A clustering algorithm such as k-means clustering, Gaussian mixture models (GMMs), k-nearest neighbors, etc. may be applied to the characteristic vectors to identify k clusters [199-205] elaborate on the matter [FIG.19 in conjunction with FIG.20B] show in response to the natural language similarity satisfying a threshold, combine the first text portion and the second text portion to generate a third text portion) encode the third text portion to define a third vector different from the first vector and the second vector; (Sun [180] feature extractor 1963 converts the training data into a plurality of text regions that can be transformed or converted to a representation on which a clustering algorithm can be performed. For example, a text-to-vector representation made be used to convert text regions into vectors that can be compared in vector space. By transforming text regions into a vector space, any of various clustering techniques may be used to identify clusters of training data that are “near” each other in a given vector space. [181] language embedding techniques may transform text into a vector space where semantically similar text appears closer in vector space than does semantically dissimilar text and/or where text with similar content appears closer in vector space than does text with dissimilar content. In this manner, text regions that are not diagnostically relevant may transform into vectors that tend to cluster in vector space and text regions that do include diagnostically relevant information (e.g., relevant from a billing perspective) may also tend to cluster in vector space [187] the vector is initialized to a predetermined value on the first iteration (e.g., the first word of a text region may be input to the language embedding model along with a vector input initialized to a predetermined value) and the vector is modified by the language embedding model in response to the input word, thus being transformed to the output vector that is fed back to the input of the model on the next iteration. [193-201] elaborate on encode the third text portion to define a third vector different from the first vector and the second vector [FIG.19 in conjunction with FIG.20B] shows the corresponding visual) update telemetry data based on the related search results and the query, the telemetry data included in training data for training at least one of the first machine- learning model or the second machine-learning model; and trigger re-training of at least one of the first machine-learning model or the second machine-learning model based on the updated telemetry data. (Sun [FIG.19 in conjunction with FIG.20B] show a visual of a loop and repetitive process for training and re-training the corresponding models based on search/query results and corresponding data which can be interpreted as "telemetry" data [0034] FIGS. 20A and 20B are block diagrams of a DLR component comprising a DLR model trained using a first stage to train a language embedding model and a second stage to produce a cluster model, in accordance with some embodiments; [0035] FIG. 21 is a flowchart illustrating a method of training a language embedding model, in accordance with some embodiments; [0077] Alternatively or additionally, in some embodiments a fact extraction component may make use of one or more statistical models to extract semantic entities from natural language input. In general, a statistical model can be described as a functional component designed and/or trained to analyze new inputs based on probabilistic patterns observed in prior training inputs. In this sense, statistical models differ from “rule-based” models, which typically apply hard-coded deterministic rules to map from inputs having particular characteristics to particular outputs. By contrast, a statistical model may operate to determine a particular output for an input with particular characteristics by considering how often (e.g., with what probability) training inputs with those same characteristics (or similar characteristics) were associated with that particular output in the statistical model's training data. To supply the probabilistic data that allows a statistical model to extrapolate from the tendency of particular input characteristics to be associated with particular outputs in past examples, statistical models are typically trained (or “built”) on large training corpuses with great numbers of example inputs. [0088] In some embodiments, given the extracted features and manual entity labels for the entire training corpus as input, the statistical entity detection model may be trained to be able to probabilistically label new texts (e.g., texts not included in the training corpus) with automatic entity labels using the same feature extraction technique that was applied to the training corpus. In other words, by processing the input features and manual entity labels of the training corpus, the statistical model may learn probabilistic relationships between the features and the entity labels.[0178] Training data 1895 may be used to establish a rules-based DLR component 1865, train a statistics-based DLR component 1865 (e.g., to train a statistical model, machine learning model, etc.), provide a combination of both rules-based and statistics-based, or otherwise train a DLR component...[182-187] describe a loop and repetitive process for training and re-training the corresponding models based on search/query results and corresponding data which can be interpreted as "telemetry" data) Sun lacks explicitly and orderly teaching execute a second machine learning model based on a query to obtain related search results corresponding to the third vector; However Steedman teaches execute a second machine learning model based on a query to obtain related search results corresponding to the third vector (Steedman [0022] receiving a user inputted query; [0023] representing the user inputted query as a sequence of embedding vectors using a first model; [0024] encoding the sequence of embedding vectors to produce a context vector using a second model; [0025] retrieving responses with associated response vectors; [0026] scoring response vectors against the context vector, wherein the scoring is a measure of the similarity between the context vector and a response vector; [0029] wherein the second model has been trained using corresponding queries and responses such that an encoding is used that maximises the similarity between the response vector and context vector for a corresponding query and response.[0049] encoding each first sequence of embedding vectors to produce a context vector using a second model; [0050] representing each response as a second sequence of vectors using a third model, wherein the third model is configured to segment an inputted response into a sequence of units from the vocabulary of units and represent each unit in the sequence as an embedding vector, wherein the third model uses at least some of the parameters of the first model...[0071] The second model 207 is trained to encode the sequence of embeddings representing a user query 201 into an output vector referred to as a context vector and shown as h.sub.X in the figure. In use, the second model receives a sequence of token embeddings D from the first model and outputs a context vector h.sub.X. Parameters of the second model may be stored in the storage 107 for example, and moved to the working memory 111 to be executed [0072] The first model 205 and second model 207 together may be referred to as an encoder 206, the encoder 206 being configured to encode the user query 201 into a context vector h.sub.X. [71-74] elaborate on the matter [FIG.2&7] show corresponding visual) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to take all prior methods and make the addition of Steedman in order to efficiently create a better output via another layer of filtering/processing via a specialized second model (Steedman [0023] representing the user inputted query as a sequence of embedding vectors using a first model; [0024] encoding the sequence of embedding vectors to produce a context vector using a second model; [0025] retrieving responses with associated response vectors; [0026] scoring response vectors against the context vector, wherein the scoring is a measure of the similarity between the context vector and a response vector [0030] The disclosed system addresses a technical problem tied to computer technology and arising in the realm of computer networks, namely the technical problem of resource utilization. The system achieves this by providing a model which is configured to segment a user inputted query into a sequence of units from a vocabulary of units and represent each unit in the sequence as an embedding vector, wherein at least one of the units in the vocabulary is an incomplete word, and wherein the first model comprises parameters that are stored using eight bits per parameter. Quantising the stored parameters using eight bits enables the model size to be reduced. Subword representation further requires a reduced vocabulary, and therefore a reduced number of embeddings to be stored. The model is therefore memory efficient and training efficient, while maintaining performance in a response selection task. Having a smaller model in terms of the number of parameters and storage required means that the model is more memory efficient and can be stored on small devices, e.g., mobile phones or tablets, with limited memory storage for example.[0146] The output of the final block 603 is taken as input to a two headed self-attention layer 613. The two self attention heads each compute weights for a weighted sum, which is scaled by the square root of the sequence length in the square-root-of-N reduction layer 615. The use of two headed attention improves the model's ability to focus on different positions compared to single headed attention, whilst still being relatively quick and efficient to train [175]the second model is configured to...This increases training speed [203] training is reduced, so that fitting larger batches may be fed into GPU memory, thus speeding up the training process.)
Corresponding product claim 8 is rejected similarly as claim 1 above. Additional Limitations: computer readable medium capable of reading and executing instructions (Sun [FIG.1 &9] show computer readable medium capable of reading and executing instructions[0208] In this respect, it should be appreciated that one implementation of embodiments of the present invention comprises at least one computer-readable storage medium (i.e., a tangible, non-transitory computer-readable medium, such as a computer memory, a floppy disk, a compact disk, a magnetic tape, or other tangible, non-transitory computer-readable medium) encoded with a computer program (i.e., a plurality of instructions), which, when executed on one or more processors, performs above-discussed functions of embodiments of the present invention. The computer-readable storage medium can be transportable such that the program...)
Corresponding method claim 15 is rejected similarly as claim 1 above.
Regarding claim 2, Sun and Steedman teach The apparatus of claim 1, wherein the programmable circuitry is to execute a third machine-learning model based on the related search results to output rankings of the related search results for presentation on a computing device. (Steedman [0050] using a third model, wherein the third model is configured to segment an inputted response into a sequence of units from the vocabulary of units and represent each unit in the sequence as an embedding vector, wherein the third model uses at least some of the parameters of the first model; [0051] encoding each second sequence of embedding vectors to produce a response vector using a fourth model; and [0052] jointly training the first, second, third and fourth models using the condition that the similarity between the context vector and the response vector is higher for a corresponding response and query and that the similarity between the context vector and the response vector is lower for a random response and query. [0053] the third model are the elements of the embedding vectors. In an embodiment, the fourth model uses at least some of the parameters of the second model. [0177] FIG. 8 shows a schematic illustration of the dual encoder system comprising the second model 207. The dual encoder further 701 comprises a third model 703. The third model 703 comprises the tokenisation algorithm 501, and uses the same stored vocabulary 509 and embeddings as the first model 205. The vocabulary 509 and the embeddings may be stored in the storage 107 and accessed by both the first model 205 and the third model 703. The third model 703 and the first model 205 thus share the embeddings, i.e. trainable parameters. The dual encoder further 701 comprises a fourth model 705. [177-183] further elaborate on the matter [FIG.7&8] show corresponding visual)
Corresponding product claim 9 is rejected similarly as claim 2 above.
Corresponding method claim 16 is rejected similarly as claim 2 above.
Regarding claim 3, Sun and Steedman teach The apparatus of claim 2, wherein the rankings are based on similarities between first ones of the related search results and second ones of the related search results. (Steedman [0040] produce a context vector using a second model, wherein the second model has been trained using corresponding queries and responses such that an encoding is used that maximises the similarity between the response vector and the context vector for a corresponding query and response; [0041] retrieve responses with associated response vectors; [0042] score response vectors against the context vector wherein the scoring is a measure of the similarity between the context vector and a response vector; and [0043] select the responses with the closest response vectors, [0044] an output, configured to output speech or text corresponding to the selected responses;[0052] jointly training the first, second, third and fourth models using the condition that the similarity between the context vector [172] the first model 205 is trained together with the second model 207. This means that the values of the embeddings used by the first model 205 and the parameters of the second model 207 are optimised. An input text is provided in S502. In this example, the input text is “new conver$ation”. The input text is tokenised in S503 using the greedy matching approach as described in relation to FIG. 5(a) and the “Inference” stage of FIG. 5(b) above. In step S504, units that are present in the vocabulary 509 are assigned their respective embeddings, so in this example, units such as “this”, “cony”, “er”, “ation” are assigned their respective embeddings. 00V characters such as “$” is assigned to one of K additional embeddings in step S505. K is a hyperparameter, and may be selected. In an embodiment, K=1000. [0176] The first model 205 and second model 207 are jointly trained as part of a dual encoder model...[227-236] elaborate on the matter [FIG.2 &7] show corresponding visual)
Corresponding product claim 10 is rejected similarly as claim 3 above.
Corresponding method claim 17 is rejected similarly as claim 3 above.
Regarding claim 4, Sun and Steedman teach The apparatus of claim 1, wherein the telemetry data includes at least one of frequencies corresponding to processed queries or metadata corresponding to the processed queries. (Sun [0089] training the ... model may involve learning, for each extracted feature, a probability with which tokens having that feature are associated with each entity type. For example, for the suffix feature “-itis,” the trained statistical entity detection model may store a probability p1 that a token with that feature should be labeled as being part of a “Problem” entity, a probability p2 that a token with that feature should be labeled as being part of a “Medication” entity, etc. ...such probabilities may be learned by determining the frequency with which tokens having the “-itis” feature were hand-labeled with each different entity label in the training corpus...[0095] train the statistical relation model by extracting features from the text, and probabilistically associating the extracted features with the manually supplied labels. Any suitable set of features may be used, as aspects of the invention are not limited in this respect. For example, in some embodiments, features used by a statistical relation model may include entity (e.g., fact type) labels, parts of speech, parser features, N-gram features, token window size (e.g., a count of the number of words or tokens present between two tokens that are being related to each other), and/or any other suitable features. [194 & 205] elaborate on the matter)
Regarding claim 5, Sun and Steedman teach The apparatus of claim 1, wherein the programmable circuitry is to arrange the first text portion and the second text portion based on the natural language similarity. (Sun [0011] a respective plurality of text regions, each having been assigned at least one medical billing code, the training data further comprising feedback from the at least one user indicating whether each medical billing code was correctly and/or incorrectly assigned, transforming each of the plurality of text regions of each of the plurality of texts to a respective representation to provide a plurality of representations, clustering the plurality of representations, and labeling each cluster [0099] extracted from input text to their standard forms). For example, in some embodiments, the tokens identified in the text as corresponding to a medical fact may be matched to corresponding terms in an ontology. In some embodiments, a list of closest matching terms may be generated, and may be ranked by their similarity to the tokens in the text. The similarity may be scored in any suitable way. For example, in one suitable technique, one or more tokens in the text may be considered as a vector of its component elements, such as words, and each of the terms in the ontology may also be considered as a vector of component elements such as words. Similarity scores between the tokens may then be computed by comparing the corresponding vectors, e.g., by calculating the angle between the vectors, or a related measurement such as the cosine of the angle. In some embodiments, one or more concepts that are linked in the ontology to one or more of the higher ranking terms (e.g., the terms most similar to the identified tokens in the text) may then be identified as hypotheses for the medical fact to be extracted from that portion of the text. Exemplary techniques that may be used in some embodiments are described in Salton, Wong, & Yang: “A vector space model for automatic indexing [0185] Thus, features 2067 provided to language embedding model may be words grouped into paragraphs (or an estimate of paragraphs) or words grouped into text regions based on proximity or one or more other factors or criteria...[188-196] elaborate on the matter)
Corresponding product claim 12 is rejected similarly as claim 5 above.
Corresponding method claim 19 is rejected similarly as claim 5 above.
Regarding claim 6, Sun and Steedman teach The apparatus of claim 1, wherein the threshold is a first threshold, and the programmable circuitry is to trigger the re-training when a quantity corresponding to the training data satisfies a second threshold. (Sun [185] train the language embedding model in an iterative manner. According to some embodiments, feedback 2075′ is initialized to some value (e.g., a pre-determined or an arbitrary value) for the first iteration for a corresponding feature 2067 or at the beginning of training.[0188] Each word in a text region is input, in turn (e.g., successively), to the language embedding model and a corresponding vector is output and fed back as input to the language embedding model on the next iteration until the last word in the text region has been input to the language embedding model (2135). According to some embodiments, the process of providing each word in a text region to the language embedding model is repeated a desired number of times (e.g., until the output vector converges) before moving to the next text region in the training data. According to some embodiments, however, each word in a given text region may be provided as input to the language embedding model a single time before applying the next text region to the language embedding model (e.g., by repeating acts 2120 and 2130 on the next of the plurality of text regions). This process may be repeated for each of the plurality of text regions in the training data to train the language embedding model [0205] multiple nearest clusters may be identified and evaluated when determining whether to exclude a text region. For example, if the nearest cluster is inconclusive (e.g., near one or more threshold values), the second (or third) nearest cluster may be evaluated to provide a more definitive answer.[FIG.19 in conjunction with FIG.20B] shows the process which can include wherein the threshold is a first threshold, and the programmable circuitry is to trigger the re-training when a quantity corresponding to the training data satisfies a second threshold.)
Corresponding product claim 13 is rejected similarly as claim 6 above.
Corresponding method claim 20 is rejected similarly as claim 6 above.
Regarding claim 7, Sun and Steedman teach The apparatus of claim 1, wherein the programmable circuitry is to determine the natural language similarity by determining a cosine similarity between the first text portion and the second text portion. (Sun [0099] Similarity scores between the tokens may then be computed by comparing the corresponding vectors, e.g., by calculating the angle between the vectors, or a related measurement such as the cosine of the angle. In some embodiments, one or more concepts that are linked in the ontology to one or more of the higher ranking terms (e.g., the terms most similar to the identified tokens in the text [0197] The nearest cluster may be identified according to which cluster has a representative vector that is closest to the characteristic vector (e.g., the Euclidean distance, cosine distance, etc., between the cluster representative vector and the characteristic vector of the text region being evaluated). [0203] In act 2230, the characteristic vector is compared to a cluster model, for example, a cluster model similar to cluster model 2073 described in connection with FIG. 20B. According to some embodiments, comparing the characteristic vector to the cluster model includes identifying one or more nearest clusters. As discussed above, identifying the nearest cluster(s) may include determining a distance (e.g., a Euclidean distance, cosine distance, etc.) between the characteristic vector and the representative vector (e.g., centroid or mean vector) for each of the clusters to identify which cluster(s) the characteristic vector is nearest. Identifying the nearest cluster(s) ...)
Corresponding product claim 14 is rejected similarly as claim 7 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARYAN D TOUGHIRY whose telephone number is (571)272-5212. The examiner can normally be reached Monday - Friday, 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aleksandr Kerzhner can be reached at (571) 270-1760. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARYAN D TOUGHIRY/Examiner, Art Unit 2165
Prosecution Insights