Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: "an obtaining unit" and "a generating unit" in claim 19.
Because this/these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 19 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim limitation 19 invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed function. Specification paragraph [0078] The obtaining unit 41 obtains information from the memory unit 30 … [0087] The generating unit 44 generates a variety of information… There is no disclosure of any particular structure, either explicitly or inherently, to perform the function stated above as claim limitation of claim 19. The specification does not provide sufficient details such that one of ordinary skill in the art would understand which filter structure or structures perform(s) the claimed function. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 15-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Shin et al. (US 20230394232 A1, hereinafter "Shin").
Regarding claim 1:
Shin discloses: An information processing method executed by a computer, the information processing method comprising:
obtaining learning data which is to be used in learning of a model that treats, as input, a plurality of sets of input information containing post-conversion information obtained by conversion of information other than a sentence text representing a sentence; (Paragraph [0038] In this example, a dataset is obtained 402. The dataset may contain a plurality of data entries and may also include structured or hierarchical data, such as information from a relational database, table, or the like. It should be appreciated that various forms of structure within one or more text documents may also correspond to structured or hierarchical data. By way of example and without limitation, heads and subheadings, internal links within documents, and the like may be a form of structured data that may provide useful information regarding how information is related within a document. In at least one embodiment, the dataset may be converted to plain text 404. Paragraph [0039] In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
performing learning using the learning data and generating the model to which information other than the sentence text is input table upon conversion. (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Regarding claim 2:
Shin discloses: The information processing method according to claim 1, further comprising
obtaining the learning data that is to be used in learning of the model to which a plurality of texts including a text obtained by conversion of information other than the sentence text is input as the plurality of sets of input information, and (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
performing learning using the learning data and generating the model to which information other than the sentence text is input table as a text. (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Regarding claim 3:
Shin discloses: The information processing method according to claim 2, further comprising
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts obtained by conversion, into text, of each set of tabular data containing information other than the sentence text, and (Paragraph [0022] In at least one embodiment, one or more automated systems may be deployed within the conversion system 104 in order to receive the input data and convert the input data into the training data 106. For example, a table-to-text conversion application may be deployed in order to identify a table, recognize different headers within the table, group different components with the associated headers, and then present the data within the table in a plain text format.)
generating the model that treats, as input, the plurality of texts based on the tabular data. (Paragraph [0028] The input manager 210 may submit a command to a data processing unit 214 to convert information from one or more datasets 216 into text-based information that may be utilized to train the language model 212. For example, the datasets 216 may include various databases or other types of structured data that include hierarchical information … such as information that is stored in a table or with submenus.)
Regarding claim 4:
Shin discloses: The information processing method according to claim 2, further comprising generating the model representing a language model to which information other than the sentence text is input table as a text. (Paragraph [0017] Embodiments provide a language model that is pre-trained on converted plain text data. For example, the text data may be acquired from different structured or relational databases or data sources and processed to preserve that associated structure, while also being converted to plain text for training purposes.)
Regarding claim 5:
Shin discloses: The information processing method according to claim 2, further comprising
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including the sentence text, and (Fig. 3E and paragraph [0037] FIG. 3E illustrates an example record 380 for the set 300 after being processed by one or more systems and methods of the present disclosure, such as the conversion system 104 and/or the data processing unit 214, among other options. For example, different micro-records of the set 382 may include a phrase or sentence that includes information obtained from evaluation of the set 300.)
generating the model that treats, as input, the plurality of texts including the sentence text. (Fig. 3E and paragraph [0037] For example, different micro-records of the set 382 may include a phrase or sentence that includes information obtained from evaluation of the set 300… Various embodiments of the present disclosure may include the generation of these micro-records as training data to pre-train one or more language models.)
Regarding claim 6:
Shin discloses: The information processing method according to claim 5, further comprising
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including the sentence text posted on Internet, and (Paragraph [0024] As previously indicated, traditional language models may be pre-trained on large corpuses of unannotated text using a denoising objective (e.g., a masking words within sentences and phrases). Unlike systems and methods of the present disclosure, these techniques ignore the structure of the input dataset. As a result, input data is limited to text-like data, such as web pages, books, articles, etc. However, additional information may be found in a variety of locations, such as structured or relational databases (e.g., SQL, XML, RRF). Various embodiments leverage the effort that has already been put into forming these databases by maintaining the structure and relations in the text. As a result, pre-training the language model 108 on this improved training data 106 provides improved results responsive to a query. Paragraph [0038] In this example, a dataset is obtained 402. The dataset may contain a plurality of data entries and may also include structured or hierarchical data, such as information from a relational database, table, or the like. It should be appreciated that various forms of structure within one or more text documents may also correspond to structured or hierarchical data. By way of example and without limitation, heads and subheadings, internal links within documents, and the like may be a form of structured data that may provide useful information regarding how information is related within a document. Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model that treats, as input, the plurality of texts including the sentence text. (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Regarding claim 15:
Shin discloses: The information processing method according to claim 1, further comprising performing learning using information other than the sentence and generating a base model and a fine-tuned model which is obtained by fine-tuning the base model to be applicable to a predetermined task. (Paragraph [0023] These language models 108 may be referred to as base language models that can be applicable for a large number of tasks and then may be specifically trained to specialized or otherwise focus on a particular domain. Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain…language models may be pre-trained on specific domains for targeted tasks.)
Regarding claim 16:
Shin discloses: The information processing method according to claim 15, further comprising generating the base model, and the fine-tuned model that has a different input of input information than the base model.
(Paragraph [0017] This language model may then be used to process information within one or more different domains and then provide responses to various input queries. Paragraph [0022] Accordingly, the system may be tuned or otherwise have a set of instructions to handling different types of input data.)
Regarding claim 17:
Shin discloses: The information processing method according to claim 15, further comprising generating the base model, and the fine-tuned model that has same order of input of input information as the base model. (Paragraph [0018] Various embodiments may evaluate these relationships and then generate one or more records associated with the relationships within the table, which may correspond to plain text information that may be useful for training purposes. As an example, a database may include information related to a class of pharmaceuticals, a method of action, and a list of compounds that drive that method of action. Upon evaluation, plain text information such as “Compound A is a type of A activator.” or “Compound A is classified as a B pharmaceutical.” may be generated. These relationships may then be utilized in pre-training a language model so that in-domain searching may be improved by leveraging the structural relationships also provided within the evaluated databases. (Indicates the language model can handle the query input that has same order or relationship as the training data))
Regarding claim 18:
Shin discloses: The information processing method according to claim 15, further comprising using label information as input information in learning of the base model. (Paragraph [0042] The plain text may also include labels or other indicators that represent the relationships. In embodiments, the plain text may be used to provide a natural language representation of at least one relationships between two or more data entries of the dataset. Accordingly, the work and effort put into forming the relationships within the database is maintained when the dataset is used for training purposes, thereby providing richer information.
Regarding claim 19:
Shin discloses: An information processing apparatus comprising:
an obtaining unit that obtains learning data which is to be used in learning of a model that treats, as input, a plurality of sets of input information containing post- conversion information obtained by conversion of information other than a sentence text representing a sentence; and (Paragraph [0038] In this example, a dataset is obtained 402. The dataset may contain a plurality of data entries and may also include structured or hierarchical data, such as information from a relational database, table, or the like. It should be appreciated that various forms of structure within one or more text documents may also correspond to structured or hierarchical data. By way of example and without limitation, heads and subheadings, internal links within documents, and the like may be a form of structured data that may provide useful information regarding how information is related within a document. Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
a generating unit that performs learning using the learning data and generates the model to which information other than the sentence text is input table upon conversion. (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Regarding claim 20:
Shin discloses: A non-transitory computer-readable storage medium having stored therein an information processing program that causes a computer to execute: (Paragraph [0097] In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein.)
obtaining learning data which is to be used in learning of a model that treats, as input, a plurality of sets of input information containing post-conversion information obtained by conversion of information other than a sentence text representing a sentence; and (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
performing learning using the learning data and generating the model to which information other than the sentence text is input table upon conversion. (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Shin in view of Radhakrishna et al. (US 20250060944 A1, hereinafter " Radhakrishna").
Regarding claim 7:
Shin teaches: The information processing method according to claim 6, further comprising
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0024] As previously indicated, traditional language models may be pre-trained on large corpuses of unannotated text using a denoising objective (e.g., a masking words within sentences and phrases). Unlike systems and methods of the present disclosure, these techniques ignore the structure of the input dataset. As a result, input data is limited to text-like data, such as web pages, books, articles, etc. However, additional information may be found in a variety of locations, such as structured or relational databases (e.g., SQL, XML, RRF). Various embodiments leverage the effort that has already been put into forming these databases by maintaining the structure and relations in the text. As a result, pre-training the language model 108 on this improved training data 106 provides improved results responsive to a query. Paragraph [0038] In this example, a dataset is obtained 402. The dataset may contain a plurality of data entries and may also include structured or hierarchical data, such as information from a relational database, table, or the like. It should be appreciated that various forms of structure within one or more text documents may also correspond to structured or hierarchical data. By way of example and without limitation, heads and subheadings, internal links within documents, and the like may be a form of structured data that may provide useful information regarding how information is related within a document. Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model that treats, as input, the plurality of texts including a text (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose explicitly obtaining the learning data including a text corresponding to a question posted on Internet.
However, Radhakrishna teaches:
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text corresponding to a question posted on Internet (Paragraph [0031] In some examples, the data extraction system 200 may extract data (e.g., text data) to train a LLM 220. The LLM 220 may be trained using a corpus of text data scraped from websites on the internet. In some cases, the corpus of text data may be extracted from Q&A forums where users may ask questions (e.g., within a question portion 205 of a Q&A post) and other users may provide answers or responses to the questions (e.g., within a response portion 210 of the Q&A post). As such, the data extraction system 200 may extract a Q&A forum post that may include the question portion 205 and the response portion 210.)
generating the model that treats, as input, the plurality of texts including a text corresponding to the question. (Paragraph [0033] In some cases, the LLM 220 may be pre-trained with a set of natural language questions and identified inputs and expected outputs (e.g., input and expected output data with corresponding labels). In other cases, the LLM 220 may be trained by extracting question portions 205 from Q&A posts and identify the corresponding input and expected output of the corresponding question portion 205.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking texts as input to train the model disclosed by Shin with fetching online Q & A forum questions to train LLM disclosed by Radhakrishna. One of the ordinary skill in the art would have been motivated to make this modification in order to allow for a more efficient generation of code (Paragraph [0014] Such techniques may allow for an LLM to be trained using natural language from Q&A posts and may allow for a more efficient generation of code, Radhakrishna).
Regarding claim 8:
Shin teaches: The information processing method according to claim 6, further comprising
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0024] As previously indicated, traditional language models may be pre-trained on large corpuses of unannotated text using a denoising objective (e.g., a masking words within sentences and phrases). Unlike systems and methods of the present disclosure, these techniques ignore the structure of the input dataset. As a result, input data is limited to text-like data, such as web pages, books, articles, etc. However, additional information may be found in a variety of locations, such as structured or relational databases (e.g., SQL, XML, RRF). Various embodiments leverage the effort that has already been put into forming these databases by maintaining the structure and relations in the text. As a result, pre-training the language model 108 on this improved training data 106 provides improved results responsive to a query. Paragraph [0038] In this example, a dataset is obtained 402. The dataset may contain a plurality of data entries and may also include structured or hierarchical data, such as information from a relational database, table, or the like. It should be appreciated that various forms of structure within one or more text documents may also correspond to structured or hierarchical data. By way of example and without limitation, heads and subheadings, internal links within documents, and the like may be a form of structured data that may provide useful information regarding how information is related within a document. Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model that treats, as input, the plurality of texts including a text (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose explicitly obtaining the learning data including a text corresponding to an answer posted on Internet.
However, Radhakrishna teaches:
obtaining the learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text corresponding to an answer posted on Internet, and (Paragraph [0031] In some examples, the data extraction system 200 may extract data (e.g., text data) to train a LLM 220. The LLM 220 may be trained using a corpus of text data scraped from websites on the internet. In some cases, the corpus of text data may be extracted from Q&A forums where users may ask questions (e.g., within a question portion 205 of a Q&A post) and other users may provide answers or responses to the questions (e.g., within a response portion 210 of the Q&A post). As such, the data extraction system 200 may extract a Q&A forum post that may include the question portion 205 and the response portion 210.)
generating the model that treats, as input, the plurality of texts including a text corresponding to the answer. (Paragraph [0036] In some examples, the LLM 220 may be used to extract the code segment 215 from a response portion 210 of a Q&A post.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking texts as input to train the model disclosed by Shin with fetching online Q & A forum responses to train LLM disclosed by Radhakrishna. One of the ordinary skill in the art would have been motivated to make this modification in order to allow for a more efficient generation of code (Paragraph [0014] Such techniques may allow for an LLM to be trained using natural language from Q&A posts and may allow for a more efficient generation of code, Radhakrishna).
Claims 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Shin in view of Coulson et al. (US 20020120806 A1, hereinafter "Coulson").
Regarding claim 9:
Shin teaches: The information processing method according to claim 2, further comprising
obtaining learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to which (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose obtaining learning data including a text obtained by conversion of a numerical value and generating the model to which a numerical value is input table as a text.
However, Coulson teaches: conversion of a numerical value into text (Paragraph [0040] FIG. 4 depicts a flow chart of an exemplary method for converting the numeric data types of FIGS. 3a-3d. Paragraph [0046] FIG. 5 is a flow chart depicting an exemplary method for converting a non-text integer dimension member to a textual representation. The method starts at step 500 and proceeds to step 502. At step 502 the sign bit or the most significant bit of the integer is inverted. This involves changing a "1" to a "0" or a "0" to a "1". The method then proceeds to step 504 and the binary representation of the integer is converted to a hexadecimal representation. Although there are many ways to convert a binary number to a textual representation, converting the binary representation to hexadecimal permits the conversion of a four-bit portion of the binary representation to a single hexadecimal digit. For example, "0010" becomes "2" and "1101" becomes "D". The method then proceeds to step 506 and ends.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking text as input and training the learning model disclosed by Shin with converting numerical data to text disclosed by Coulson. Converting numerical data into text before model training is a well-known routine process conventional activity previously known to the industry. One of the ordinary skill in the art would have been motivated to make this modification in order to train the machine learning model with uniformed text input effectively and efficiently.
Regarding claim 10:
Shin teaches: The information processing method according to claim 9, further comprising
obtaining learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to which (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose obtaining learning data including a text obtained by conversion of an integer value and generating the model to which a numerical value is input table as a text.
However, Coulson teaches: conversion of an integer value into text (Paragraph [0040] FIG. 4 depicts a flow chart of an exemplary method for converting the numeric data types of FIGS. 3a-3d. Paragraph [0046] FIG. 5 is a flow chart depicting an exemplary method for converting a non-text integer dimension member to a textual representation. The method starts at step 500 and proceeds to step 502. At step 502 the sign bit or the most significant bit of the integer is inverted. This involves changing a "1" to a "0" or a "0" to a "1". The method then proceeds to step 504 and the binary representation of the integer is converted to a hexadecimal representation. Although there are many ways to convert a binary number to a textual representation, converting the binary representation to hexadecimal permits the conversion of a four-bit portion of the binary representation to a single hexadecimal digit. For example, "0010" becomes "2" and "1101" becomes "D". The method then proceeds to step 506 and ends.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking text as input and training the learning model disclosed by Shin with converting an integer value to text disclosed by Coulson. Converting numerical data including integer into text before model training is a well-known routine process conventional activity previously known to the industry. One of the ordinary skill in the art would have been motivated to make this modification in order to train the machine learning model with uniformed text input effectively and efficiently.
Regarding claim 11:
Shin teaches: The information processing method according to claim 9, further comprising
obtaining learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to which (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose obtaining learning data including a text obtained by conversion of a real number and generating the model to which a numerical value is input table as a text.
However, Coulson teaches: conversion of a floating-point number into text (Paragraph [0040] FIG. 4 depicts a flow chart of an exemplary method for converting the numeric data types of FIGS. 3a-3d. Paragraph [0047] FIG. 6 is a flow chart depicting an exemplary method for converting a non-text floating-point dimension member to a textual representation. The method begins at step 600 and proceeds to decision block 602. At decision block 602, a determination is made as to whether the floating-point dimension member is positive. If the floating-point number is not positive, then the method branches to step 608. At step 608, all of the bits of the floating-point number are inverted. The method then proceeds to step 606, wherein the binary representation of the floating-point number is converted to a hexadecimal representation. The method then proceeds to step 610 and ends. Paragraph [0048] Returning now to decision block 602, if a determination is made that the floating-point dimension member is positive, the method branches to step 604 and the most significant bit or the sign bit is inverted. The method then proceeds to step 606 and the binary representation of the floating-point number is converted to a hexadecimal representation. The method then proceeds to step 610 and ends.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking text as input and training the learning model disclosed by Shin with converting a float-point number to text disclosed by Coulson. Converting numerical data including real numbers into text before model training is a well-known routine process conventional activity previously known to the industry. One of the ordinary skill in the art would have been motivated to make this modification in order to train the machine learning model with uniformed text input effectively and efficiently.
Regarding claim 12:
Shin teaches: The information processing method according to claim 9, further comprising
obtaining learning data to be used in learning of the model that treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose obtaining learning data including a text obtained by conversion of the numerical value indicating date and time at which the sentence was posted.
However, Coulton teaches the conversion of the numerical value indicating date and time into text (Paragraph [0031] As can be seen in FIG. 2, an integer value 210 of "1" is converted by the non-text to text conversion 204 into a text representation 250. Similarly, the textual representation 250 is converted by the reverse-conversion 206 to integer representation 230. The same conversion and reverse-conversion can be performed on the remaining dimension member data 212-224. Namely, single-precision (32 bit) floating-point numbers 212, double-precision (64 bit) floating-point numbers 214, decimal numbers 216, dates 218, text strings 220, Boolean values 222, and null values 224 can be converted by an exemplary embodiment of the present invention to text format and reverse-converted to the applicable native format. Paragraph [0032] The textual dimension member data 250-262 depicted in FIG. 2 also include codes 290 specifying the class type of the dimension member (e.g., numeric, date, string) and sub-codes 292 specifying the sub-class of the dimension member (e.g., integer, floating-point, decimal). The non-text to text conversion 204 assigns a code 290 to each dimension member and a sub-code 292 to each numeric dimension member during the conversion. While the codes 290 enable the differentiation of types, such as numeric types and string types, the sub-codes 292 enable the differentiation of sub-types, such as floating-points and decimals. By classifying the dimension member data, sorts can be more narrowly tailored. The order generated when sorting textual dimension member data can be adjusted by selecting the codes used for the different types. For example, using code "0" for numeric and code "2" for strings will cause numeric values to be sorted before string values. The sub-codes allow numeric values to be sorted independently of their type. When converting a numeric value for a dimension with mixed data types the value is first converted to a double precision floating point number. That double precision floating point number is converted to text to form the first part of the string. Then a code for the sub-type is appended. If the precision of the value is more than can be stored in a double precision floating point number, then the original value is converted to text to form the last part of the string. In the examples in the database file 202, this last portion is shown for all the numeric types 250-256, because this portion is only applicable to numeric values.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking text as input and training the learning model disclosed by Shin with converting a numerical formatted date and time to text disclosed by Coulson. Converting numerical data including date and time into text before model training is a well-known routine process conventional activity previously known to the industry. One of the ordinary skill in the art would have been motivated to make this modification in order to train the machine learning model with uniformed text input effectively and efficiently.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Shin in view of Wang et al. (US 20230139831 A1, hereinafter "Wang").
Regarding claim 13:
Shin teaches: The information processing method according to claim 2, further comprising
obtaining learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin does not specifically disclose obtaining learning data including a test obtained by conversion of a character string other than the sentence, and generating the model to which a character string other than the sentence is input table as a text.
However, Wang teaches conversion of a character string other than the sentence into plain text (Paragraph [0060] To process an image containing both machine-typed and handwritten characters, the document conversion engine 220 first separates the document into lines of text in step 724, using the techniques described herein (e.g., with respect to step 716). Paragraph [0061] In step 732, each line of text is converted to text using appropriate methods, e.g., OCR for printed text, and the trained handwriting recognition model for handwritten text.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking converted text as input and training the learning model disclosed by Shin with converting character string into text for downstream tasks disclosed by Wang. One of the ordinary skill in the art would have been motivated to make this modification in order to extract keywords from a document via comparing characters in the words. (Paragraph [0038] For example, for an individual tax return form, first the form can be segmented into several parts based on keywords present in the document for each section, and then every item in each section is converted to text and compared with pre-defined keywords that are required to be extracted, and items are selected as intended information based on the comparison result. The comparison can include various text analytic and neutral language processing methods, such as to compare characters in the words or the semantic meaning of the words, Wang).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Shin in view of Wang, and further in view of Cho et al. (US 20150201058 A1, hereinafter "Cho").
Regarding claim 14:
Shin teaches: The information processing method according to claim 13, further comprising
obtaining learning data that is to be used in learning of the model which treats, as input, the plurality of texts including a text (Paragraph [0039] In at least one embodiment, the dataset may be converted to plain text 404. In various embodiments, the plain text may also be preferred to as a plain text corpus. For example, as shown in FIGS. 3A-3E, information within the dataset may be extracted and then relationships within the dataset may be labeled or otherwise presented as plain text. Such conversion(s) may include removing the information from a table or a database and then linking or otherwise textually describing relationships between different information from the dataset. That is, the converted plain text corpus may include natural language representations of at least one relationships between two or more data centuries of the data entries forming the data set. In this manner, the dataset may be converted to a format that is useful for training machine learning systems.)
generating the model to (Paragraph [0040] The converted dataset, in the form of plain text (e.g., the plain text corpus), may be used to update one or more parameters of a language model 406. For example, the language model may execute in a training mode and receive the plain text corpus as an input. The information may be used to adjust various weighting parameters of the language model and/or to fine tune the language model for a particular domain.)
Shin in view of Wang do not specifically disclose obtaining learning data including a text obtained by conversion of the character string indicating day of week on which the sentence was posted, and generating the model to which day of week on which the sentence was posted is input table as a text.
However, Cho teaches the conversion of the character string indicating day of the week into text (Paragraph [0111] The mobile terminal according to the present invention can convert at least one of the user voice inputted through the microphone 122 and the counterpart voice received through the wireless communication unit 110 to a text [i.e., STT (speech to text) conversion] [S501]. The controller 180 extracts meaningful information from the converted text [S502]… And, the meaningful information may include at least one of character identification information, place information and time information. Paragraph [0112] For further instance, if information estimated as time information such as a date, a day of the week, a time and the like is discovered from the converted text, the controller 180 extracts the information discovered from the converted text and is then able to save the extracted information in the memory 160.)
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of taking character string converted text as input and training the learning model disclosed by Shin in view of Wang with converting character string indicating day of the week associated with the content into text during the speech-to-text process disclosed by Cho. One of the ordinary skill in the art would have been motivated to make this modification in order to take the best use of the time information such as registering the time information as a memo. (Paragraph [0114] For further instance, if the information extracted during the call includes time information, the applications associable with the time information may include an alarm application for setting an alarm for a time indicated by the time information, schedule information for registering a schedule taking place at a time indicated by the time information, a memo application for registering the time information as a memo, and the like, Cho).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WENNA DUAN whose telephone number is (571)270-5795. The examiner can normally be reached Monday-Friday (8: 30 am - 5:00 pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ABDULLAH KAWSAR can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.D./Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
