Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim[ 5, 12, 19 ] are objected to because of the following informalities: Claims 5, 12, and 19 recite “based a zero-shot-react-description”, where they should describe “based on a zero-shot-react-description”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim [1-20] are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ),
second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim [1] recites, “selecting, by the computer program, a dynamic template database to the
information retrieved by the large language model based on the retrieved information”. This limitation is not clear in its current form, as it isn’t clear as to what is actually happening based on the description of a system selecting a database to the information retrieved by the large language model. Based on the specification, at ¶ [0037], it should read something along the lines of, “selecting, by the computer program, a dynamic template from a dynamic template database based on the retrieved information”.
Claim 1 also recites “the terms” in line 15. “ A term” is introduced in the earlier lines of the claim. However, the claim does not introduce “the terms” in the plural form. Therefore, it is unclear whether “the terms” refers to the previously recited “a term”, to multiple terms, or to some other set of terms.
Claims [8] and [15] include the same language, and are indefinite for the same reasons as claim [1].
Additionally, claims 5, 12, and 19 recite “the dynamic template”. Claim 1, 8, 15 do not introduce “a dynamic template”, therefore, it’s unclear what “the dynamic template” refers to.
Claims [2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 16, 17,18, 19, 20] are rejected due to dependency on
Claim 1, 8, and 15.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim [1-20] rejected under 35 U.S.C. 101 because the claimed invention is directed to a mental
process without significantly more.
Regarding claim 1, 8, 15 recite a method, a non-transitory computer readable storage medium and a system comprising:
receiving, by a
tokenizing, by the
selecting, by the
querying, by the
and outputting, by the
As described above, these limitations can be carried out as a series of mental steps by a person.
In addition to the mental steps, claim 1, 8 and 15 describe an electronic device and a large language model which are additional elements. These are hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
This judicial exception is not integrated into a practical application because the only additional
element recited are “electronic device” and “a large language model” and these additional elements are nothing more than general-purpose software. These claims do not include any additional elements that amount to significantly more than a mental process, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 2, 9, 16 recite a method, system and a non-transitory computer readable medium including instruction, further comprising: using a term sheet sample to infer a dynamic template and select the dynamic template from the dynamic template database [person can use their mental process to use some sample data and select a template];
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 9 and 16 describe using hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as described above, the only additional elements recited are, a memory, a processor, and a computer program executing on a user device, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 3, 10, 17 recite a method, system and a non-transitory computer readable medium including instruction, further comprising extracting a rule for generating a new term from the extracted term [ person can use their mental process to generate a new term based on rules].
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 10 and 17 describe using hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as described above, the only additional elements recited are, a memory, a processor, and a computer program executing on a user device, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 4, 11, 18 recite a method, system and a non-transitory computer readable medium including instruction, further comprising wherein the querying step occurs more than once, collating the output for each query – [person can query the appropriate document based on user input using their mental process again and gather the output data];
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 11 and 18 describe using hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as described above, the only additional elements recited are, a memory, a processor, and a computer program executing on a user device, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 5, 12, 19 recite a method, system and a non-transitory computer readable medium including instruction, further comprising further comprising selecting the dynamic template based a zero-shot-react-description.– [person can use their mental process to intuitively solve problems using a combination of mental reasoning and active information gathering]
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 12 and 19 describe using hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as described above, the only additional elements recited are, a memory, a processor, and a computer program executing on a user device, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 6, 13, 20 recite a method, system and a non-transitory computer readable medium including instruction further comprising generating a new dynamic template based on the terms and a meta instruction [ person can use their mental process to generate a new template based on terms and meta data for providing context and meaning to the information].
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 13 and 20 describe using hardware (memory, hardware processor or computer processor) and software components. These are all general-purpose hardware and software being used as a tool to implement the abstract idea.
These claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, as described above, the only additional elements recited are, a memory, a processor, and a computer program executing on a user device, and these additional elements are nothing more than instructions to apply the mental process using general-purpose software and hardware.
Regarding Claim 7, 14 recite a method and a non-transitory computer readable medium including instruction further comprising post-processing the terms into a chart. [ person can use their mental process to generate a chart for visual analysis].
As described above, these limitations can be carried out as a series of mental steps. In addition to the mental steps, claim 14 describes using instruction in a software component. This is a software tool to implement the abstract idea; thus, the claims do not describe a practical application or significantly more than the mental process.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims [1, 3, 4, 6, 8, ,10, 11, 13, 15,17,18, 20] are rejected under 35 U.S.C. 103 as being
unpatentable over Li (US Patent No US20230237570) in view of Maschmeyer (US 20240256792 A1)
and in further view of Hodges (US. Patent No. US 20090276695 A1)
Regarding claim 1, Li teaches a method, comprising:
receiving, by a computer program executed on an electronic device, a brief description of a term sheet in a natural language format from a user interface connected to the electronic device – [0023 “The present invention further leverages artificial intelligence, natural language processing, and machine learning technology to facilitate the analysis and generation of contracts. The various system applications that leverage artificial intelligence, natural language processing, and machine learning technology may be trained and configured for proper practice of the invention prior to a user or entity beginning the process of contract analysis and generation. Additionally, said various system applications may be further trained and improved as the user or entity managing the contract analysis and generation system processes a plethora of legal contracts and other legal documents ripe for analysis using the system”]; [0039 “The entity system(s) 604 and user device(s) 601 include data storage and processing capabilities for storing data related to the system environment, but not limited to, data created and/or used by (as shown in FIG. 8) a plurality of system applications 801. As shown in FIG. 8, the various applications of the contract analysis and generation system, and their related data, may be held in a memory 800 of one or more entity system(s) 604 and one or more user device(s) 601. In some embodiments, the user may interact with a contract analysis and generation system through a user interface 802”]; [0006 “The present invention overcomes these drawbacks by using a contract analysis and generation process and system which functions, generally, as follow: Natural language processing (“NLP”) and artificial intelligence (“AI”) are used to transform relevant business files with text into question lists and contract templates. These generated question lists and contract templates may then be used by business negotiators to efficiently and effectively discuss key contract points during offline negotiations”]; [0007 “In one embodiment, a method for contract analysis and generation comprises processing a file resulting in a processed file, parsing keywords from said processed file resulting in a plurality of parsed keywords, generating a question list based on the parsed keywords, generating a contract based on a previously created contract template and said question list, approving and signing of said contract by all relevant parties resulting in a signed contract, and uploading said signed contract to the machine learning and data analysis module for archiving and future model training purposes”].
However, Li does not teach tokenizing, by the computer program, the brief description to create a plurality of numerical values for a content of the brief description and using, by the computer program, a large language model to retrieve information relevant to the brief description based on the tokenization;
However, Maschmeyer teaches tokenizing, by the computer program, the brief description to create a plurality of numerical values for a content of the brief description – [0064 “An example of how the transformer 50 may process textual input data is now described. Input to a language model (whether transformer-based or otherwise) typically is in the form of natural language as may be parsed into tokens. It should be appreciated that the term “token” in the context of language models and NLP has a different meaning from the use of the same term in other contexts such as data security. Tokenization, in the context of language models and NLP, refers to the process of parsing textual input (e.g., a character, a word, a phrase, a sentence, a paragraph, etc.) into a sequence of shorter segments that are converted to numerical representations referred to as tokens (or “compute tokens””].
And Maschmeyer also teaches using, by the computer program, a large language model to retrieve information relevant to the brief description based on the tokenization- [0060 “A language model may use a neural network (typically a DNN) to perform natural language processing (NLP) tasks such as language translation, image captioning, grammatical error correction, and language generation, among others. A language model may be trained to model how words relate to each other in a textual sequence, based on probabilities. A language model may contain hundreds of thousands of learned parameters or in the case of a large language model (LLM) may contain millions or billions of learned parameters or more”]; [0065 “In general, the token sequence that is inputted to the transformer 50 may be of any length up to a maximum length defined based on the dimensions of the transformer 50 (e.g., such a limit may be 2048 tokens in some LLMs). Each token 56 in the token sequence is converted into an embedding vector 60 (also referred to simply as an embedding). An embedding 60 is a learned numerical representation (such as, for example, a vector) of a token that captures some semantic meaning of the text segment represented by the token 56. The embedding 60 represents the text segment corresponding to the token 56 in a way such that embeddings corresponding to semantically-related text are closer to each other in a vector space than embeddings corresponding to semantically-unrelated text. For example, assuming that the words “look”, “see”, and “cake” each correspond to, respectively, a “look” token, a “see” token, and a “cake” token when tokenized, the embedding 60 corresponding to the “look” token will be closer to another embedding corresponding to the “see” token in the vector space, as compared to the distance between the embedding 60 corresponding to the “look” token and another embedding corresponding to the “cake” token. The vector space may be defined by the dimensions and values of the embedding vectors. Various techniques may be used to convert a token 56 to an embedding 60. For example, another trained ML model may be used to convert the token 56 into an embedding 60. In particular, another trained ML model may be used to convert the token 56 into an embedding 60 in a way that encodes additional information into the embedding 60 (e.g., a trained ML model may encode positional information about the position of the token 56 in the text sequence into the embedding 60). In some examples, the numerical value of the token 56 may be used to look up the corresponding embedding in an embedding matrix 58 (which may be learned during training of the transformer 50)” (emphasis added)].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li into the teaches of Maschmeyer because combining the document generation system of Li with the techniques of Maschmeyer where the techniques use retrieving a token and then looking up the corresponding embedding in a matrix would improve the system’s ability to interpret user provided natural language descriptions. Language models that operate on tokenized text and numerical embeddings provide improved semantic understanding of user input, enabling more accurate identification of document elements and improved retrieval of relevant document information to create legal documents.
Li in view of Maschmeyer doesn’t teach selecting, by the computer program, a dynamic template database to the information retrieved by the large language model based on the retrieved information; and querying, by the computer program using the large language model, the dynamic template database using the retrieved information to extract a term; and outputting, by the computer program, the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term.
However, Hodges teaches selecting, by the computer program, a dynamic template database to the information retrieved by the large language model based on the retrieved information- [0100 “The automatic document generator 2 also comprises a data receiving module 18 which is arranged to receive data from a user 4. Typically, a user will upload data to the data receiving module 18, which data are intended to be displayed in a document. The data receiving module 18 is connected to a template selection tool 20 which in turn is connected to the template storage unit 14. The template selection tool 20 is arranged to receive data from the data receiving module 18 and to search through the templates stored in the template storage unit 14 in order to select the most appropriate template for the data that are input”].
and querying, by the computer program using the large language model, the dynamic template database using the retrieved information to extract a term; and – [0005 “According an aspect of the present invention there is provided a template selector comprising: a data receiving module for receiving input data which are intended for display in a document; a template storage unit for storing data relating to a plurality of document templates; and a template selection tool for selecting an appropriate template from the template storage unit by comparing at least one attribute of the input data to at least one attribute of each of the stored templates”]; [0006 “In this way the template that best fits the input data may be selected and used as the basis for building a document”]; [0052 “Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described ; [0075 “Preferably a document generator is provided that comprises the template selector defined above and a document building module for building a document by fitting the input data to the created template. In this way, a template can be created and a document generated based only on input data that are intended for display in a document. The created template can be built from a base which is most suited to the input data and from there it may be tailored to best match the specific parameters of the input data”].
and outputting, by the computer program, the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term - [0036 “The data may have some form of default formatting before they are formatted by the document building module. Thus, the document building module may be arranged to re-format the input data”]; [0052” Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described”]; [“[0112 “At generation step 58, the user can send an instruction to the automatic document generator 2 for it to generate a document. The document produced by the automatic document generator 2 may be previewed by the user at document preview step 60.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer to the teachings of Hodges because incorporating the techniques of Hodges in order to generate structured documents using predefined template. Applying Hodges’ template-based document generation to the legal document generation system of Li in view of Maschmeyer would advantageously enable consistent formatting, automated organization of document content, and more efficient generation of legal documents based on user input.
Regarding claim 3, Li does teach the method of clam 1, further comprising extracting a rule for generating a new term from the extracted term – [ 0040 “The user interface 802 is further configured to perform single click contract generation. The AI trained contract clause repository 146 contains a plurality of contract clauses and is configured to check whether all important clauses are contained within a contract. The AI-trained default clause repository 214 uses an AI-based model trained on questions to determine if questions have default answers and is configured to insert said default answers next to their respective questions in a question list. The rules of contract wording module 116 contains keyword rules specific to properly wording contracts. The AI model trained on contract wording 117 is an AI-based model trained to parse contract terms from a question list. The non-keyword repository 118 is configured to remove keywords parsed from the AI named-entity recognition (NER) that are not considered as variables in a contract from the text on which the system is operating (typically a list of keywords). The AI model trained to generate questions 126 is an AI-based model trained on keywords and questions and is configured to convert sentences containing keywords into questions written in ordinary language. The AI model trained to classify questions 127 is an AI-based model trained on questions and classification categories and is configured to classify questions into various question categories. The machine learning data analysis module 40 is configured to perform the following functions as needed: comparing different templates and different versions of a template, comparing a contract with a contract template, comparing different versions of a contract with one another, and comparing contracts with previously signed contracts which used the same or a similar template. The party management module 50 may include, but is not limited to, enterprise resource planning (ERP) and CRM systems utilized by users and entities of the system, and is configured to insert party information in question lists and contract templates. The party management module 50 may further be communicatively coupled to the memory 800 and the user interface 802. The contract lifecycle management module 60 can be communicatively coupled to the memory 800 and the user interface 802 and configured to facilitate the management of a contract's lifecycle, including, but not limited to, alerting users to the necessity of new contracts, reminding users of upcoming contract expiration dates, and managing amendments made during the execution process of the contract”]; [0042 “The process of template generation 20 uses a first question list 13 to generate a first contract template 25, a standard contract question list 23, and a standard contract template 24. The standard contract template 24 will be later used during the step of contract drafting 30. The template generation process 300 may utilize a party management module 50 (which may include, but is not limited to, ERP and CRM systems utilized by users and entities of the system) to insert party information into (as shown in FIG. 4) the standard question list 23 and the standard contract template 24, effectively generating a second question list with party information 32 and a second contract template with party information 33. Next, offline negotiation 34 occurs. There, various users and entities participating in business negotiations utilize the second question list 32 and the second contract template 33 to efficiently fill in the answer columns in the second question list 32 resulting in a completed second question list. This completed second question list 32 is herein referred to as a third question list 333. In some embodiments, offline contract negotiations may be guided by the machine learning data analysis module 40. Once negotiations are complete and said third question list 333 is then uploaded 35 by a user 602 to one or more entity system(s) 604 or one or more user device(s) 601, a first contract 36 may be generated by a single click”].
Regarding claim 4, Li in view Maschmeyer does not teach the method of claim 1, further comprising, wherein the querying the step occurs more than once, collating the output for each query.
But Hodges teaches generating documents by retrieving a template and building a document. The generated document is reasonably interpreted as a filled template. Hodges’s further discloses transmitting the generated document to storage systems such as a temporary storage unit or content management system, there by indicating that multiple filled template (generated documents) maybe be created and stored when multiple sets of data are processed. When multiple sets of input data are processed, the system would inherently generate multiple documents from templates, which would correspond to multiple filled templates. – [0034 “According to another aspect of the present invention there may be provided a document generator comprising: the template selector as previously described and a document building module for building a document by fitting the input data to the selected template”]; [“[0078 “According to yet another aspect of the present invention there is provided a computer readable storage medium having stored there on a computer program, the computer program comprising: a program module which receives input data, which data are intended for display in a document; and a program module which selects an appropriate template from a template storage unit by comparing at least one attribute of the input data to at least one attribute of each of a plurality of templates stored in the template storage unit”]; [0005 “According an aspect of the present invention there is provided a template selector comprising: a data receiving module for receiving input data which are intended for display in a document; a template storage unit for storing data relating to a plurality of document templates; and a template selection tool for selecting an appropriate template from the template storage unit by comparing at least one attribute of the input data to at least one attribute of each of the stored templates”]. [0107 ” The document building module 24 sends finished routed documents as electronic files such as PDF, EPS, or other, either directly to the CMS 30 or initially to the spooler 32 and then to the CMS 30”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer to the teachings of Hodges’s so that document templates could be dynamically created again or adjusted again based on characteristics of the input data. Such a modification would advantageously allow the system to automatically generate templates that better fit the content being produced, thereby improving the flexibility, efficiency, and accuracy of automated generated documents.
Regarding claim 6, Li in view Maschmeyer in view of Hodges does teach the method of claim 1, further comprising, generating a new dynamic template based on the terms
Li in view Maschmeyer does not teach the method of claim 1, further comprising, generating a new dynamic template based on a meta instruction.
But Hodges’s teaches generating a template based on a meta instruction - [0033 “Preferably templates are stored in the template storage unit as meta-data. The meta-data may comprise information about the templates such as the number of objects therein and data relating to the objects such as their area and associated data acceptance rules. Thus, the data capacity of the template storage unit may be minimized. Also, the computational burden of searching through the template storage unit may be minimized. By using meta-data, it is an abstraction of a real template that may be used in the template selection process”]; [0111 “At channel selection step 56, the user may specify the form of output that is desired. For example, the user may specify whether a soft or hard copy should be produced, the resolution that is required, and the quality of paper that should be used “]; [0112 “At generation step 58, the user can send an instruction to the automatic document generator 2 for it to generate a document. The document produced by the automatic document generator 2 may be previewed by the user at document preview step 60. An optional final route step may be provided in which the user can instruct the automatic document generator 2 to publish the document in the manner specified at the channel selection step 56. [0113 ” The InDesign Plugin 29 utilizes program functions for interfacing between an external server (not shown) and the automatic document generator 2. The external server comprises software for generating documents which is accessible via the InDesign Plugin 29”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer to the teachings of Hodges’s because generating templates using the user input and meta instruction to define desired output formats allows systems to automatically understand how to handle, or display data based on metatags. This helps in saving time and increasing productivity and allows structured output to be displayed. It also allows for rapid scaling of content and reduces repetitive manual work leading to better higher quality results.
Regarding claim 8, Li discloses a non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computers cause the one or more computers to perform steps comprising: receiving, by a computer program executed on an electronic device - [0025 “The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing”]
receiving, by a computer program executed on an electronic device a brief description of a term sheet in a natural language format from a user interface connected to the electronic device - [0023 “The present invention further leverages artificial intelligence, natural language processing, and machine learning technology to facilitate the analysis and generation of contracts. The various system applications that leverage artificial intelligence, natural language processing, and machine learning technology may be trained and configured for proper practice of the invention prior to a user or entity beginning the process of contract analysis and generation. Additionally, said various system applications may be further trained and improved as the user or entity managing the contract analysis and generation system processes a plethora of legal contracts and other legal documents ripe for analysis using the system”]; [0039 “The entity system(s) 604 and user device(s) 601 include data storage and processing capabilities for storing data related to the system environment, but not limited to, data created and/or used by (as shown in FIG. 8) a plurality of system applications 801. As shown in FIG. 8, the various applications of the contract analysis and generation system, and their related data, may be held in a memory 800 of one or more entity system(s) 604 and one or more user device(s) 601. In some embodiments, the user may interact with a contract analysis and generation system through a user interface 802”]; [0006 “The present invention overcomes these drawbacks by using a contract analysis and generation process and system which functions, generally, as follow: Natural language processing (“NLP”) and artificial intelligence (“AI”) are used to transform relevant business files with text into question lists and contract templates. These generated question lists and contract templates may then be used by business negotiators to efficiently and effectively discuss key contract points during offline negotiations”]; [0007 “In one embodiment, a method for contract analysis and generation comprises processing a file resulting in a processed file, parsing keywords from said processed file resulting in a plurality of parsed keywords, generating a question list based on the parsed keywords, generating a contract based on a previously created contract template and said question list, approving and signing of said contract by all relevant parties resulting in a signed contract, and uploading said signed contract to the machine learning and data analysis module for archiving and future model training purposes”].
However, Li does not teach tokenizing by the computer program, the brief description to create a plurality of numerical values for a content of the brief description and using by a computer program a large language model to retrieve information relevant to the brief description based on the tokenization.
However, Maschmeyer teaches tokenizing the brief description to create a plurality of numerical values for a content of the brief description – [0064 “An example of how the transformer 50 may process textual input data is now described. Input to a language model (whether transformer-based or otherwise) typically is in the form of natural language as may be parsed into tokens. It should be appreciated that the term “token” in the context of language models and NLP has a different meaning from the use of the same term in other contexts such as data security. Tokenization, in the context of language models and NLP, refers to the process of parsing textual input (e.g., a character, a word, a phrase, a sentence, a paragraph, etc.) into a sequence of shorter segments that are converted to numerical representations referred to as tokens (or “compute tokens””].
And Maschmeyer also teaches using by a computer program, a large language model to retrieve information relevant to the brief description based on the tokenization- [0003 “A large language model (LLM) is a deep learning algorithm that can process natural language to summarize, translate, predict and generate text and other content. A LLM may be trained to learn billions of parameters in order to model how words relate to each other in a textual sequence. Inputs to a LLM may be referred to as prompts. A prompt is a natural language input that includes instructions to cause the LLM to generate a desired output”]; [0060 “A language model may use a neural network (typically a DNN) to perform natural language processing (NLP) tasks such as language translation, image captioning, grammatical error correction, and language generation, among others. A language model may be trained to model how words relate to each other in a textual sequence, based on probabilities. A language model may contain hundreds of thousands of learned parameters or in the case of a large language model (LLM) may contain millions or billions of learned parameters or more.
[0065 “In FIG. 1B, a short sequence of tokens 56 corresponding to the text sequence “Come here, look!” is illustrated as input to the transformer 50. Tokenization of the text sequence into the tokens 56 may be performed by some pre-processing tokenization module such as, for example, a byte pair encoding tokenizer (the “pre” referring to the tokenization occurring prior to the processing of the tokenized input by the LLM), which is not shown in FIG. 1B for simplicity. In general, the token sequence that is inputted to the transformer 50 may be of any length up to a maximum length defined based on the dimensions of the transformer 50 (e.g., such a limit may be 2048 tokens in some LLMs). Each token 56 in the token sequence is converted into an embedding vector 60 (also referred to simply as an embedding). An embedding 60 is a learned numerical representation (such as, for example, a vector) of a token that captures some semantic meaning of the text segment represented by the token 56. The embedding 60 represents the text segment corresponding to the token 56 in a way such that embeddings corresponding to semantically-related text are closer to each other in a vector space than embeddings corresponding to semantically-unrelated text. For example, assuming that the words “look”, “see”, and “cake” each correspond to, respectively, a “look” token, a “see” token, and a “cake” token when tokenized, the embedding 60 corresponding to the “look” token will be closer to another embedding corresponding to the “see” token in the vector space, as compared to the distance between the embedding 60 corresponding to the “look” token and another embedding corresponding to the “cake” token. The vector space may be defined by the dimensions and values of the embedding vectors. Various techniques may be used to convert a token 56 to an embedding 60. For example, another trained ML model may be used to convert the token 56 into an embedding 60. In particular, another trained ML model may be used to convert the token 56 into an embedding 60 in a way that encodes additional information into the embedding 60 (e.g., a trained ML model may encode positional information about the position of the token 56 in the text sequence into the embedding 60). In some examples, the numerical value of the token 56 may be used to look up the corresponding embedding in an embedding matrix 58 (which may be learned during training of the transformer 50).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li into the teaches of Maschmeyer because combining the document generation system of Li with the techniques of Maschmeyer would improve the system’s ability to interpret user provided natural language descriptions. Language models that operate on tokenized text and numerical embeddings provide improved semantic understanding of user input, enabling more accurate identification of document elements and improved retrieval of relevant document information to create legal documents.
Li in view of Maschmeyer doesn’t teach selecting by a computer program, a dynamic template database to the information retrieved by the large language model based on the retrieved information; and querying the computer program using the large language model, the dynamic template database using the retrieved information to extract a term; and outputting by a computer program, the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term.
However, Hodgess teaches selecting a dynamic template database to the information retrieved by the large language model based on the retrieved information- [0100 “The automatic document generator 2 also comprises a data receiving module 18 which is arranged to receive data from a user 4. Typically, a user will upload data to the data receiving module 18, which data are intended to be displayed in a document. The data receiving module 18 is connected to a template selection tool 20 which in turn is connected to the template storage unit 14. The template selection tool 20 is arranged to receive data from the data receiving module 18 and to search through the templates stored in the template storage unit 14 in order to select the most appropriate template for the data that are input”].
and querying by the computer program, using the large language model, the dynamic template database using the retrieved information to extract a term; and – [0005 “According an aspect of the present invention there is provided a template selector comprising: a data receiving module for receiving input data which are intended for display in a document; a template storage unit for storing data relating to a plurality of document templates; and a template selection tool for selecting an appropriate template from the template storage unit by comparing at least one attribute of the input data to at least one attribute of each of the stored templates”]; [0006 “In this way the template that best fits the input data may be selected and used as the basis for building a document”]; [0052 “Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described.
and outputting, by the computer program, the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term - [0036 “The data may have some form of default formatting before they are formatted by the document building module. Thus, the document building module may be arranged to re-format the input data”]; [0052” Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described”]; [“[0112 “At generation step 58, the user can send an instruction to the automatic document generator 2 for it to generate a document. The document produced by the automatic document generator 2 may be previewed by the user at document preview step 60.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer to the teachings of Hodges’s because incorporating the techniques of Hodges’s in order to generate structured documents using predefined template. Applying Hodges’s template-based document generation to the legal document generation system of Li in view of Maschmeyer would advantageously enable consistent formatting, automated organization of document content, and more efficient generation of legal documents based on user input.
Regarding claim 10, the instructions of claim 8, further comprising, extracting a rule for generating a new term from the extracted term. Claim 10 is rejected for the same reasons as claim 3.
Regarding claim 11, the instructions of claim 8, further comprising, wherein the querying step occurs more than once, collating the output for each query. Claim 11 is rejected for the same reasons as claim 4.
Regarding claim 13, the instructions of claim 8, further comprising generating a new dynamic template based on the terms and meta instruction. Claim 13 is rejected for the same reasons as claim 6.
Regarding claim 15, Li discloses a computer processing system comprising: a memory configured to store instructions; and a hardware processor operatively coupled to the memory for executing the instructions to – [0034 “As used herein the term “user device” may refer to any device that employs a processor and memory and can perform computing functions, such as a personal computer or a mobile device, wherein a mobile device is any mobile communication device, such as a cellular telecommunications device (i.e., a cell phone or mobile phone), a mobile Internet accessing device, or other mobile device. and a hardware processor operatively coupled to the memory for executing the instructions to - [0031 “It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions”].
receive a brief description of a term sheet in a natural language format from a user interface connected to the electronic device - [0023 “The present invention further leverages artificial intelligence, natural language processing, and machine learning technology to facilitate the analysis and generation of contracts. The various system applications that leverage artificial intelligence, natural language processing, and machine learning technology may be trained and configured for proper practice of the invention prior to a user or entity beginning the process of contract analysis and generation. Additionally, said various system applications may be further trained and improved as the user or entity managing the contract analysis and generation system processes a plethora of legal contracts and other legal documents ripe for analysis using the system”]; [0039 “The entity system(s) 604 and user device(s) 601 include data storage and processing capabilities for storing data related to the system environment, but not limited to, data created and/or used by (as shown in FIG. 8) a plurality of system applications 801. As shown in FIG. 8, the various applications of the contract analysis and generation system, and their related data, may be held in a memory 800 of one or more entity system(s) 604 and one or more user device(s) 601. In some embodiments, the user may interact with a contract analysis and generation system through a user interface 802”]; [0006 “The present invention overcomes these drawbacks by using a contract analysis and generation process and system which functions, generally, as follow: Natural language processing (“NLP”) and artificial intelligence (“AI”) are used to transform relevant business files with text into question lists and contract templates. These generated question lists and contract templates may then be used by business negotiators to efficiently and effectively discuss key contract points during offline negotiations”]; [0007 “In one embodiment, a method for contract analysis and generation comprises processing a file resulting in a processed file, parsing keywords from said processed file resulting in a plurality of parsed keywords, generating a question list based on the parsed keywords, generating a contract based on a previously created contract template and said question list, approving and signing of said contract by all relevant parties resulting in a signed contract, and uploading said signed contract to the machine learning and data analysis module for archiving and future model training purposes”].
However, Li does not teach tokenize the brief description to create a plurality of numerical values for a content of the brief description and use a large language model to retrieve information relevant to the brief description based on the tokenization.
However, Maschmeyer teaches tokenize the brief description to create a plurality of numerical values for a content of the brief description – [0064 “An example of how the transformer 50 may process textual input data is now described. Input to a language model (whether transformer-based or otherwise) typically is in the form of natural language as may be parsed into tokens. It should be appreciated that the term “token” in the context of language models and NLP has a different meaning from the use of the same term in other contexts such as data security. Tokenization, in the context of language models and NLP, refers to the process of parsing textual input (e.g., a character, a word, a phrase, a sentence, a paragraph, etc.) into a sequence of shorter segments that are converted to numerical representations referred to as tokens (or “compute tokens””].
And Maschmeyer also teaches use of a large language model to retrieve information relevant to the brief description based on the tokenization- [0003 “A large language model (LLM) is a deep learning algorithm that can process natural language to summarize, translate, predict and generate text and other content. A LLM may be trained to learn billions of parameters in order to model how words relate to each other in a textual sequence. Inputs to a LLM may be referred to as prompts. A prompt is a natural language input that includes instructions to cause the LLM to generate a desired output”]; [0060 “A language model may use a neural network (typically a DNN) to perform natural language processing (NLP) tasks such as language translation, image captioning, grammatical error correction, and language generation, among others. A language model may be trained to model how words relate to each other in a textual sequence, based on probabilities. A language model may contain hundreds of thousands of learned parameters or in the case of a large language model (LLM) may contain millions or billions of learned parameters or more.
[0065 “In FIG. 1B, a short sequence of tokens 56 corresponding to the text sequence “Come here, look!” is illustrated as input to the transformer 50. Tokenization of the text sequence into the tokens 56 may be performed by some pre-processing tokenization module such as, for example, a byte pair encoding tokenizer (the “pre” referring to the tokenization occurring prior to the processing of the tokenized input by the LLM), which is not shown in FIG. 1B for simplicity. In general, the token sequence that is inputted to the transformer 50 may be of any length up to a maximum length defined based on the dimensions of the transformer 50 (e.g., such a limit may be 2048 tokens in some LLMs). Each token 56 in the token sequence is converted into an embedding vector 60 (also referred to simply as an embedding). An embedding 60 is a learned numerical representation (such as, for example, a vector) of a token that captures some semantic meaning of the text segment represented by the token 56. The embedding 60 represents the text segment corresponding to the token 56 in a way such that embeddings corresponding to semantically-related text are closer to each other in a vector space than embeddings corresponding to semantically-unrelated text. For example, assuming that the words “look”, “see”, and “cake” each correspond to, respectively, a “look” token, a “see” token, and a “cake” token when tokenized, the embedding 60 corresponding to the “look” token will be closer to another embedding corresponding to the “see” token in the vector space, as compared to the distance between the embedding 60 corresponding to the “look” token and another embedding corresponding to the “cake” token. The vector space may be defined by the dimensions and values of the embedding vectors. Various techniques may be used to convert a token 56 to an embedding 60. For example, another trained ML model may be used to convert the token 56 into an embedding 60. In particular, another trained ML model may be used to convert the token 56 into an embedding 60 in a way that encodes additional information into the embedding 60 (e.g., a trained ML model may encode positional information about the position of the token 56 in the text sequence into the embedding 60). In some examples, the numerical value of the token 56 may be used to look up the corresponding embedding in an embedding matrix 58 (which may be learned during training of the transformer 50).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li into the teaches of Maschmeyer because combining the document generation system of Li with the techniques of Maschmeyer would improve the system’s ability to interpret user provided natural language descriptions. Language models that operate on tokenized text and numerical embeddings provide improved semantic understanding of user input, enabling more accurate identification of document elements and improved retrieval of relevant document information to create legal documents.
Li in view of Maschmeyer doesn’t teach select a dynamic template database to the information retrieved by the large language model based on the retrieved information; and query the computer program using the large language model, the dynamic template database using the retrieved information to extract a term; and outputting the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term.
However, Hodges teaches select a dynamic template database to the information retrieved by the large language model based on the retrieved information- [0100 “The automatic document generator 2 also comprises a data receiving module 18 which is arranged to receive data from a user 4. Typically, a user will upload data to the data receiving module 18, which data are intended to be displayed in a document. The data receiving module 18 is connected to a template selection tool 20 which in turn is connected to the template storage unit 14. The template selection tool 20 is arranged to receive data from the data receiving module 18 and to search through the templates stored in the template storage unit 14 in order to select the most appropriate template for the data that are input”].
and query the large language model, the dynamic template database using the retrieved information to extract a term; and – [0005 “According an aspect of the present invention there is provided a template selector comprising: a data receiving module for receiving input data which are intended for display in a document; a template storage unit for storing data relating to a plurality of document templates; and a template selection tool for selecting an appropriate template from the template storage unit by comparing at least one attribute of the input data to at least one attribute of each of the stored templates”]; [0006 “In this way the template that best fits the input data may be selected and used as the basis for building a document”]; [0052 “Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described.
and output the terms in a determined format based on the dynamic template database, the format being determined based on the extracted term - [0036 “The data may have some form of default formatting before they are formatted by the document building module. Thus, the document building module may be arranged to re-format the input data”]; [0052” Once a template has been created, the template may be used for generating a document. This may be achieved by manually inputting data to the template or by automatically building a document using building rules, as previously described”]; [“[0112 “At generation step 58, the user can send an instruction to the automatic document generator 2 for it to generate a document. The document produced by the automatic document generator 2 may be previewed by the user at document preview step 60.”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer to the teachings of Hodges’s because incorporating the techniques of Hodges’s in order to generate structured documents using predefined template. Applying Hodges’s’ template-based document generation to the legal document generation system of Li in view of Maschmeyer would advantageously enable consistent formatting, automated organization of document content, and more efficient generation of legal documents based on user input.
Regarding claim 17, the system of claim 15, further comprising extracting a rule for generating a new term from the extracted term. Claim 17 is rejected for the same reasons as claim 3.
Regarding claim 18, The system of claim 15, further comprising, wherein the querying step occurs more than once, collating the output for each query. Claim 18 is rejected for the same reasons as claim 4.
Regarding claim 20, the instructions of claim 15, further comprising generating a new dynamic template based on the terms and a meta instruction. Claim 20 is rejected for the same reasons as claim 6.
Claims [2, 9, 16] are rejected under 35 U.S.C. 103 as being unpatentable over Li (US Patent
No US20230237570) in view of Maschmeyer (US 20240256792 A1) and in further view of Hodges (US.
Patent No. US 20090276695 A1) and in further view of Lagi (US 20200065857).
Regarding claim 2, Li in view of Maschmeyer and in further view of Hodge does teach the
method of claim 1,
selecting the dynamic template from the dynamic template database. Hodges teaches - [ 0100 “The
automatic document generator 2 also comprises a data receiving module 18 which is arranged to
receive data from a user 4. Typically, a user will upload data to the data receiving module 18, which data
are intended to be displayed in a document. The data receiving module 18 is connected to a template
selection tool 20 which in turn is connected to the template storage unit 14. The template selection tool
20 is arranged to receive data from the data receiving module 18 and to search through the templates
stored in the template storage unit 14 in order to select the most appropriate template for the data that
are input”].
However, Li in view of Maschmeyer and in further view of Hodge does not teach the
method of clam 1 further comprising using a term sheet sample to infer a dynamic template.
However, Lagi teaches using sample data to infer a dynamic template–
[0150 “…A message template may include fields to be filled by the content generation system 216 with
information, including content that is generated by the content generation system 216 based on
information selected from the knowledge graph 210. In some embodiments, the system may
automatically infer or generate message templates from historical data provided by the user and/or
other users of the system 200. Historical data may include, but is not limited to, historical
communication data (e.g., previously sent messages) and/or historical and current customer
relationship data, such as the dates, amounts, and attributes of business transactions. In some
embodiments, the system 200 may further rely on the objective of a message to generate the
template”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer in further view of Hodges to the teachings of Lagi in order to automatically derive document templates from historical legal documents and subsequently select and utilize those templates for generating new legal documents. Such a combination would advantageously improve efficiency in document generation, and enable system to leverage previously generated legal documents to determine appropriate templates for future document generation.
Regarding claim 9, the instructions according to claim 8, further comprising using a term sheet sample to infer a dynamic template and select the dynamic template from the dynamic template database. Claim 9 is rejected for the same reasons as claim 2.
Regarding claim 16, the system of claim 15, further comprising using a term sheet sample to infer a dynamic template and select the dynamic template from the dynamic template database. Claim 16 is rejected for the same reasons as claim 2.
Claim [5, 12, 19] are rejected under 35 U.S.C. 103 as being unpatentable are rejected under
35 U.S.C. 103 as being unpatentable over Li (US Patent No US20230237570) in view of Maschmeyer
(US20240256792 A1) and in further view of Hodges (US. Patent No. US 20090276695 A1) and in further
view of LangChain Agent Types (webpage published on 4/18/23).
Regarding claim 5, Li in view of Maschmeyer and in further view of Hodge does not teach about selecting template based on zero-shot-react description.
However, Langchain does teach about a Zero-shot-react description- [Agent types -“zero-shot-react-description: This agent uses the ReAct framework to determine which tool to use based solely on the tool’s description. Any number of tools can be provided. This agent requires that a description is provided for each tool”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer in further view of Hodges to the teachings of Langchain in order to enable the system to reason over a user request and automatically select an appropriate template. Integrating a reasoning-based agent such as described by zero-shot-react description would allow the system to reason about a request properly and then act by selecting the appropriate template for generating the desired document.
Regarding claim 12, the instructions of claim 8, further comprising selecting the dynamic template based a zero-shot-react-description. Claim 12 is rejected for the same reasons as claim 5.
Regarding claim 19, the instructions of claim 15, further comprising selecting the dynamic template based a zero-shot-react-description. Claim 19 is rejected for the same reasons as claim 5.
Claim [7, 14] are rejected under 35 U.S.C. 103 as being unpatentable are rejected under 35
U.S.C. 103 as being unpatentable over Li (US Patent No US20230237570) in view of Maschmeyer
(US20240256792 A1) and in further view of Hodges (US. Patent No. US 20090276695 A1) and in further
view of Brereton (US20140040805).
Regarding claim 7, Li in view of Maschmeyer in view of Hodge does not teach the method of
claim 1, further comprising post-processing the terms into a chart.
However, Brereton displays results as a chart- [0011 “In some example embodiments, a chart is
defined to include any type of report, table, graph, diagram, etc. that is derived from user inputs for data queries. The user can specify one or more settings for a data query, wherein the results provide an indication of the user's interest. In response, the data query module can retrieve and display these results set. Additionally, based on the state of these non-chart, based settings (the data query inputs by the user), the chart module makes inferences to determine the charts that would provide additional insight to the user. Along with the results set, the chart module can return a chart befitting the results set. Some example embodiments alleviate the need for a user to specify their interests multiple times for different types of charts and information about requested data”]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Li in view of Maschmeyer and in further view of Hodges to the teachings of Brereton because information associated with generated legal documents could be displayed in chart form. Such a modification would advantageously allow users to visually analyze structured information derived from the generated legal documents through graphical representations, thereby improving comprehension, comparison , and analysis of the document data. This allows users to have additional insight with requiring user to manually create visualizations.
Regarding claim 14, the instructions according to claim 8 further comprising post-processing the terms into a chart. Claim 14 is rejected for the same reasons as claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHEZA ABDUL AZIZ whose telephone number is (571)272-9610. The examiner can normally be reached Monday-Friday 7:30am-5pm Alternate Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL C WASHBURN/Supervisory Patent Examiner, Art Unit 2657