CTNF 18/334,752 CTNF 92093 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101. Claims 1, 10 and 18 are directed to an abstract idea. The claims are essentially a process for organizing and manipulating information: identify tokens from text, decide two token are related, merge them into a new token, and generate output from the merged token set. These are information-analysis and data-manipulation steps that can be characterized as a mental process or abstract data processing, which fall within the abstract idea workframe. The claims do not integrate the abstract idea into a practical application. They do not recite a particular machine, a specific model architecture, a technical improvement to tokenization, an improved computer operation or any concrete use of the output data. It simply says to determine tokens, determine a relationship, merge tokens, and determine output data. The framework, a claim that recites an abstract idea remains ineligible when the additional elements do not meaningfully limit the idea or apply it in a practical technological way. The claims also lack an inventive concept. The steps are stated at a high functional level and do not add unconventional technology beyond the abstract token merging concept itself. Even if the method were performed on a computer, generic computer implementation is not enough to make the claim patent eligible; Alice holds that merely implementing an abstract idea on a generic computer does not transform it into patent eligible subject matter. Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15-aia AIA Claim(s) 1-3, 5-6, 8, 10-12, 14 and 17-20 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Yemelyanenko et al. (US 2021/0334477, hereinafter Yeme) . Claims 1 and 10, Yeme teaches a method comprising ([claim 1] Yeme teaches “A computer-implemented method for processing a text sequence for a machine processing task to be performed by a Machine Learning Algorithm (MLA), the method executable by a server, the method comprising”) : determining a first set of tokens associated with text corresponding to input data ([claim 1] Yeme teaches acquiring text and splitting it into an initial token sequence: “acquiring, by the server, the text sequence indicative of at least one word”; and “using, by the server, the token vocabulary for splitting the given word into an initial token sequence, the initial token sequence representing individual characters of the given word”; Yeme also teaches that the token vocabulary stores a set of tokens: “the token vocabulary storing a set of token”) ; determining that a first token of the first set of tokens is related to a second token of the first set of tokens ([claim 1] Yeme teaches merge-table relation between token pairs: “the merge table indicating possible mergers between pairs of tokens from the set of tokens”; Yeme further teaches identifying mergeable adjacent token pairs: “using, by the server, the merge table for identifying a set of possible merges between pairs of adjacent token sequence”; [0198] [Step 404] the specification given a concrete example: “a merge between the tokens ‘a’ and ‘b’ … is possible, while a merge between tokens ‘x’ and ‘z’ is not possible”) ; determining, based at least on the first token being related to the second token, a second set of tokens by at least merging the first token with the second token to generate a third token ([claim 1] Yeme teaches merging tokens to generate a new token sequence: “using, by the server, the reduced set of possible merges for generating a new token sequence by performing at least one of the reduced set of possible merges in the current token sequence”; [0194] [Step 402] Yeme also teaches that a merged token can be formed from two tokens: “a given multi-character token may correspond to a merged token of at least two single-character tokens”) ; and determining, based at least on the second set of tokens, output data associated with the input data ([claim 1] Yeme teaches determining a final token sequence for the machine-processing task: “identifying, by the server, the current token sequence of the other given merging iteration as the final token sequence to be used for the machine processing task”; Yeme also teaches downstream output generation, including “generating translated content based thereon” and “providing the translated content to the user”) . Claims 2 and 11, Yeme further teaches the method of claim 1, wherein the determining that the first token is related to the second token comprises: determining a score associated with a relationship between the first token and the second token ([0121] Yeme teaches first identifying possible merges between token pairs: “use the merge table 230 for identifying a set of possible merges between pairs of adjacent tokens in a current token sequence”; [0133] Yeme then assigns a value to the possible merger relationship: “assign a random value to respective ones of the set of possible merges ranging from 0 to 1”; under BRI, the “random value” is a score associated with the possible merge relationship between the adjacent tokens) , and determining that the score is equal to or greater than a threshold score ([0133] Yeme teaches comparing the assigned value to a threshold: “compare these random values against a pre-determined threshold value”; Yeme then teaches the positive threshold determination: “If a given random value is above the pre-determined threshold value, the respective possible merge from the set of possible merges is included into a reduced set of possible merges”; A score “above” the threshold is greater than the threshold and therefore satisfices “equal to or greater than.”) . Claims 3 and 12, Yeme further teaches the method of claim 1, further comprising: determining that a fourth token of the first set of tokens is related to a fifth token of the first set of tokens ([0158-0161] Yeme teaches identifying additional possible merges in the same initial token sequence: For example, it identifies “the following set of possible merges,” including “r’ + ’e’ which would yield ‘re’”; [0121] Yeme also teaches using the “merge table 230 for identifying a set of possible merges between pairs of adjacent tokens in a current token sequence”; Thus, the fourth/fifth tokens may be mapped to “r” and “e”, which are determined to be related because they are identified as a possible merge.”) ; and determining, based at least on a threshold number, to refrain from merging the fourth token with the fifth token ([0133-0134] Yeme teaches threshold-based exclusion: “compare these random values against a pre-determined threshold value”; Yeme further teaches: “if a given random value is below the pre-determined threshold value, the respective possible merge from the set of possible merges is excluded from a reduced set of possible merges.”; Yeme teaches that this threshold verification is an “inclusion and/or exclusion criterion determining whether a given possible merge … is to be present in a reduced set of possible merges.”; [0167-0178] in the concrete third-attempt example, the system excludes “’r’ + ’e’ which would yield ‘re’”; [0186] The then performs “’a’ + ’t’ … thereby yielding ‘at’”, not “r” + “e””) . Claim 5, Yeme further teaches the method of claim 1, wherein: the first set of tokens includes at least the first token, the second token, and a fourth token ([0112] Yeme’s initial token sequence include “r”, “e” and “u”: “the word 302 is split into the initial token sequence 314 of ‘”u” “n” “r” “e” “l” “a” “t” “e” “d””; Mapping: first token “r”; second token “e”; fourth token “u”) ; and the second set of tokens includes at least the fourth token and the third token ([0155] Yeme’s new token sequence includes the unchanged fourth token “u” and the generated third token “re”: “generate a new token sequence 326 “”u” “n” “re” “l” “a” “t” “e” “d””; [0114] the same results is also taught in [0114]: “generate a new token sequence 316 being “u” “n” “re” “l” “a” “t” “e” “d”) . Claim 6, Yeme further teaches the method of claim 5, wherein: the fourth token is one of before the first token or after the second token within the first set of tokens ([0112] Yeme teaches first set “u” “n” “r” “e” “l” “a” “t” “e” “d”, the fourth token “u” is before the first token “r”; Yeme teaches this initial token sequence in [0112]) ; and the fourth token is the one of before the third token or after the third token within the second set of tokens ([0114] Yeme teaches second set “u” “n” “re” “l” “a” “t” “e” “d”, the same fourth token “u” is before the generated third token “re”; Yeme teaches the new token sequence in [0114] and again in [0155]) . Claim 8, Yeme further teaches the method of claim 1, wherein the first set of tokens includes at least the first token, followed by the second token, and followed by a fourth token, and wherein the method further comprising ([0112] Yeme’s initial token sequence is “u” “n” “r” “e” “l” “a” “t” “e” “d”) : determining, based at least on the first token being followed by the second token, whether the first token is related to the second token ([0113] Yeme determines whether adjacent followed-by tokens are related by identifying possible merges between adjacent token pairs: “the merge table 230 may be accessed and may be used to identify a set of possible merges between pairs of adjacent tokens in the initial token sequence 314”; for the specific first/second pair, Yeme identifies “the possible merge of adjacent tokens ‘r’ and ‘e’ into a merged token ‘re’) ; and determining, based at least on the second token being followed by the fourth token, whether the second token is related to the fourth token ([0122-0126] Yeme teaches identifies the second/fourth adjacent pair “e” + “l” as possible merge; Yeme states that “the server 106 using the merge table 230 identifies the following set of possible merges,” including “’r’ + ‘e’ which would yield ‘el’”; Thus, based on “e” being followed by “l”, the server determines whether “e” is related to “l” by checking the merge table and identifying “e” + “l” as a possible merge) . Claim 14, The system of claim 10, wherein: the first set of tokens includes a fourth token that is one of before the first token or after the second token; and the second set of tokens includes the fourth token that is the one of before the third token or after the third token. (Claim 14 contains subject matter similar to claims 5 and 6, and thus is rejected under similar rationale) Claims 17 and 20, Yeme further teaches the system of claim 10, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implementing one or more large language models (LLMs); a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources ([0012] “Machine Learning Algorithm (MLA)”) . Claim 18, Yeme teaches a processor comprising ([0084] “one or more processors”) : one or more processing units to generate, using one or more machine learning models, an output based at least on a first set of tokens associated with text corresponding to input data ([0087] Yeme teaches a sever implementing/accessing an MLA: “the server 106 may implement and/or have access to a Machine Learning Algorithm (MLA) 124”; Yeme further teaches that MLAs “make data-driven predictions or decisions expressed as outputs”; [0090] Yeme also teaches that NNs are used where “it is only important to know an output based on a given input,” including “automatic text translation into different languages”; [0155] The first set of tokens is mapped to the generated new token sequence “u” “n” “re” “l” “a” “t” “e” “d””; Yeme teaches that this “new token sequence 326” is generated and used as the current sequence for later processing; [0191] Yeme teaches that the token sequence is used for the machine-processing task: “the server 106 may be configured to identifying the given current token sequence … as a given final token sequence to be used for the machine processing task”; [0086] Yeme also teaches translation output: the server executes algorithm for “processing/preparing this content for translation”, “generating translated content based thereon,” and “providing the translated content to the user”) , wherein the first set of tokens is generated based at least on merging a first token of a second set of tokens with a second token of the second set of tokens ([0112] Yeme teaches the second set/initial token sequence: “the word 302 is split into the initial token sequence 314 of “u” “n” “r” “e” “l” “a” “t” “e” “d”; [0155] Yeme then teaches merging the first and second tokens: “the server 106 may be configured to perform the possible merge ‘r’ + ‘e’ … thereby yielding ‘re’”; The resulting first set is taught: “generate a new token sequence 326 “u” “n” “re” “l” “a” “t” “e” “d”) , the first token representing at least a first portion of the text and the second token representing at least a second portion of the text ([0112] Yeme teaches that the initial token sequence represents individual character of the word: “the conventional subword segmentation process 310 may begin with splitting the word 302 into an initial token sequence 314 that represents individual characters of the word 302”; under the mapping, first token “r” represents the “r” character portion of “unrelated” and second token “e” represents the “e” character portion of “unrelated”) . Claim 19, Yeme further teaches the processor of claim 18, wherein the one or more processing units are further to: determine a first score associated with a relationship between the first token and the second token ([0122-0125] Yeme first identifies the relationship as possible merge: “the server 106 using the merge table 230 identifies the following set of possible merges,” including “’r’ + ‘e’ which would yield ‘re’”; [0133] Yeme then determines a numerical value for each possible merge: “assign a random value to respective ones of the set of possible merges ranging from 0 to 1”; Under BRI, the random value is the claimed first score associate with the possible-merge relationship between “r” and “e”) ; and determine that the score is equal to or greater than a threshold score ([0133] Yeme teaches threshold comparison: “the server 106 may be configured to compare these random values against a pre-determined threshold value”; Yeme further teaches the positive determination: “If a given random value is above the pre-determined threshold value, the respective possible merge from the set of possible merges is included into a reduced set of possible merges”; A value “above” the threshold is greater than the threshold and satisfies the claimed “equal to or greater than” condition) , wherein the merging of the first token with the second token is based at least on the score being equal to or greater than the threshold score ([0133] Yeme teaches that above-threshold possible merges are included in the reduced set; [0155] and that the server uses that reduced set of perform the merge; Specifically, Yeme teaches that the server “may be configured to perform the possible merge ‘r’ + ‘e’ … thereby yielding ‘re’”, and to “generate a new token sequence 326 “u” “n” “re” “l” “a” “t” “e” “d””; [1090] Yeme also states that, at a given merging iteration, the server uses “the reduced set of possible merges for generating a given new token sequence by performing at least one of the reduced set of possible merges”) . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 4, 9, 13 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yemelyanenko et al. (US 2021/0334477, hereinafter Yeme) and further in view of Provilkov et al. (“BPE-Dropout: Simple and Effective Subword Regularization”; July 5-10, 2020; Association for Computational Linguistics) . Claims 4 and 13, Yeme teaches all the limitations in claim 3. The difference between the prior art and the claimed invention is that Yeme does not explicitly teach determining a first score associated with a first relationship between the first token and the second token; determining a second score associated with a second relationship between the fourth token and the fifth token; and determining that the first score is greater than the second score, wherein the determining refrain from merging the fourth token with the fifth token is further based at least on the first score being greater than the second score. Provilkov teaches determining a first score associated with a first relationship between the first token and the second token ([2.1] Provilkov teaches score/priority determination for token-pair merge relationships: BPE “iteratively counts all pairs of tokens and merges the most frequent pair into a new token”; [Introduction] [2.1] Provilkov states that the merge table specifies “the priority of the merges.”) ; determining a second score associated with a second relationship between the fourth token and the fifth token ([Algorithm 1] Provilkov’s scoring applies to all token-pairs candidates: Algorithm 1 identifies “all possible merges of tokens from current split.”; [2.1] The same BPE procedure “counts all pairs of tokens,” so a second possible merge pair likewise has a count/frequency/priority score) ; and determining that the first score is greater than the second score, wherein the determining refrain from merging the fourth token with the fifth token is further based at least on the first score being greater than the second score ([2.1] Provilkov teaches selecting the higher-scored merge: “the pair of adjacent tokens which has the highest priority is merged.”; Algorithm 1 likewise states: “select the merge with highest priority” and “apply merge to current split.”; Thus, lower/priority/lower-score candidate merges are not applied in that iteration) . Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yeme with teachings of Provilkov by modifying the method and server for processing text sequence for machine processing task as taught by Yeme to include determining a first score associated with a first relationship between the first token and the second token; determining a second score associated with a second relationship between the fourth token and the fifth token; and determining that the first score is greater than the second score, wherein the determining refrain from merging the fourth token with the fifth token is further based at least on the first score being greater than the second score as taught by Provilkov for the benefit of better learning the compositionality of words and being robust to segmentation errors (Provilkov [Abstract]) . Claims 9 and 16, Yeme teaches all the limitations in claim 1. The difference between the prior art and the claimed invention is that Yeme does not explicitly teach determining that the third token of the second set of tokens is related to a fourth token of the second set of tokens; and determining, based at least on the third token being related to the fourth token, a third set of tokens by at least merging the third token with the fourth token to generate a fifth token, wherein determining the output data associated with the input data is based at least on the third set of tokens. Provilkov teaches determining that the third token of the second set of tokens is related to a fourth token of the second set of tokens ([Fig. 1a] Provilkov’s Fig. 1a shows the second token set after the first merge as including “re-l” in the segmentation path for “unrelated”; The caption sates: “Hyphens indicate possible merges”; Mapping: third token = “re”; fourth token = “l”) ; and determining, based at least on the third token being related to the fourth token, a third set of tokens by at least merging the third token with the fourth token to generate a fifth token ([Fig. 1a] Provilkov’s Fig. 1(a) shows the path from “un re-l-at-ed” to “un rel-at-ed,” i.e., merging “re” and “l” to generate “rel”; [2.1] The text also states that “the pair of adjacent tokens which has the highest priority is merged,” and this is done “iteratively”) , wherein determining the output data associated with the input data is based at least on the third set of tokens ([Algorithm 1] Provilkov’s Algorithm 1 repeatedly applies merges to the “current split” and then “returns current split.”; [Fig. 1a] Fig. 1a shows the process continuing after “rel” toward the final “unrelated” output) . Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yeme with teachings of Provilkov by modifying the method and server for processing text sequence for machine processing task as taught by Yeme to include determining that the third token of the second set of tokens is related to a fourth token of the second set of tokens; and determining, based at least on the third token being related to the fourth token, a third set of tokens by at least merging the third token with the fourth token to generate a fifth token, wherein determining the output data associated with the input data is based at least on the third set of tokens as taught by Provilkov for the benefit of better learning the compositionality of words and being robust to segmentation errors (Provilkov [Abstract]) . 07-21-aia AIA Claim (s) 7 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yemelyanenko et al. (US 2021/0334477, hereinafter Yeme) and further in view of Corrado et al. (US 9,141,916) . Claims 7 and 15, Yeme teaches all the limitations in claim 1. The difference between the prior art and the claimed invention is that Yeme does not explicitly teach wherein the merging the first token with the second token comprises: determining a first vector associated with the first token; determining a second vector associated with the second token; and determining, based at least on the first vector and the second vector, a third vector associated with the third token. Corrado teaches wherein the merging the first token with the second token comprises: determining a first vector associated with the first token ([col. 1 line 64 to col. 2 line 17] Corrado teaches that an embedding function maps “a single token to a floating vector”; Mapping: first token = Yeme’s “r”; first vector = Corrado’s vector for a token) ; determining a second vector associated with the second token ([col. 1 line 64 to col. 2 line 17] Corrado teaches mapping “each token in a list” to a “respective floating point vector.”; Mapping: second token = Yeme’s “e”; second vector = Corrado’s respective vector for that token) ; and determining, based at least on the first vector and the second vector, a third vector associated with the third token ([col. 5 lines 12-29] Corrado teaches that a combining embedding function merges token vectors into a “single merged vector” and that the merged vector may use a “sum, average, or weighted liner combination,”; Mapping: third token = Yeme’s merged token “re”; third vector = Corrado’s merged vector based on the first and second vectors) . Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the teachings of Yeme with teachings of Corrado by modifying the method and server for processing text sequence for machine processing task as taught by Yeme to wherein the merging the first token with the second token comprises: determining a first vector associated with the first token; determining a second vector associated with the second token; and determining, based at least on the first vector and the second vector, a third vector associated with the third token include as taught by Corrado for the benefit of (Corrado [col. 2 line 59 to col. 3 line 3]) . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Xu et al. (US 2023/0161977) – Implementations of the present disclosure relate to methods, devices, and computer program products for generating a destination vocabulary from a source vocabulary. In a method, a group of candidate vocabularies are determined from the source vocabulary based on a corpus, a size of a candidate vocabulary in the group of candidate vocabularies being different from a size of the source vocabulary. A group of marginal scores are obtained for the group of candidate vocabularies, respectively, a marginal score in the group of marginal scores being obtained for the candidate vocabulary based on a corpus entropy of the candidate vocabulary and a size of the candidate vocabulary. The destination vocabulary is selected from the group of candidate vocabularies based on the group of marginal scores. With these implementations, both of the corpus entropy and the vocabulary size are considered in the vocabulary generation, and thus a balance may be achieved therebetween, which may increase the performance of the generated vocabulary. Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHREYANS A PATEL whose telephone number is (571)270-0689. The examiner can normally be reached Monday-Friday 8am-5pm PST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. SHREYANS A. PATEL Primary Examiner Art Unit 2653 /SHREYANS A PATEL/ Examiner, Art Unit 2659 Application/Control Number: 18/334,752 Page 2 Art Unit: 2659 Application/Control Number: 18/334,752 Page 3 Art Unit: 2659 Application/Control Number: 18/334,752 Page 4 Art Unit: 2659 Application/Control Number: 18/334,752 Page 5 Art Unit: 2659 Application/Control Number: 18/334,752 Page 6 Art Unit: 2659 Application/Control Number: 18/334,752 Page 7 Art Unit: 2659 Application/Control Number: 18/334,752 Page 8 Art Unit: 2659 Application/Control Number: 18/334,752 Page 9 Art Unit: 2659 Application/Control Number: 18/334,752 Page 10 Art Unit: 2659 Application/Control Number: 18/334,752 Page 11 Art Unit: 2659 Application/Control Number: 18/334,752 Page 12 Art Unit: 2659 Application/Control Number: 18/334,752 Page 13 Art Unit: 2659 Application/Control Number: 18/334,752 Page 14 Art Unit: 2659
Prosecution Insights