DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to amendments filed on 08/11/2026, in which claims 1, 3-11, 16 are amended and claims 2, 14, and 19 are canceled. Claims 1-20 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8, 11, 12, 16, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Shen (US20210035556A1) in view of Wortsman (Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time).
Regarding claim 1, Shen teaches a memory operatively coupled to the system, wherein the memory stores computer executable components; and a processor that executes the computer executable components (Par. 0091, As illustrated, the computing system 400 comprises
a processor 401 coupled to a mass storage unit 403 and accessing a working memory) and performs the following steps: automatically converting a first original dataset to a first modified dataset, wherein the first modified dataset is created in accordance with at least one requirement of a language model (LM), wherein the first modified dataset is utilized to train the LM (Par. 0093, The LM controller 407 includes a pre-processing module 413 and a language modelling (LM) module 415. The pre-processing module is configured to process inputs to prepare them for inputting into the language model (as described above). The LM module is configured to perform prediction based on the processed input. The LM controller may be configured to train the language model of the LM module in accordance with the training methods described herein; Abs, the embodiments described herein fine-tune language models for downstream tasks solely by pre-processing the training data set); applying the first modified dataset to the LM to create a first modified LM (Par. 0093, The LM controller 407 includes a pre-processing module 413 and a language modelling (LM) module 415. The pre-processing module is configured to process inputs to prepare them for inputting into the language model (as described above). The LM module is configured to perform prediction based on the processed input. The LM controller may be configured to train the language model of the LM module in accordance with the training methods described herein).
Shen fails to teach automatically converting a second original dataset to a second modified dataset; applying the second modified dataset to the LM to create a second modified LM; and fusing the first modified LM with the second modified LM to form a fused LM, wherein the fused LM comprises a combination of first features of a first individual fine-tuned model present in the first modified LM with second features of a second individual fine-tuned model present in the second modified LM, and wherein the first features and the second features comprise respective parameter weightings of the respective fine-tuned models.
Wortsman teaches (Abs, We show that averaging the weights of multiple models fine tuned with different hyperparameter configurations often improves accuracy and robustness, Pg. 3, Let θ = FineTune(θ_0,h) denote the parameters obtained by fine-tuning with pre-trained initialization θ_0 and hyperparameter configuration h. The hyperparameter configuration can include the choice of optimizer, data augmentation, training iterations, and a random seed which will determine data order……. The uniform soup is constructed by averaging all fine-tuned models θ_i, the uniform soup is the fused LM which has a combination of the features of the models fused).
Shen is analogous to the claimed invention because it is in the field of endeavor of preparing/preprocessing data for model training. Wortsman is analogous to the claimed invention because it is in the field of endeavor of fusing models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling data of the claimed invention to have used the method in Shen to generate multiple fine-tuned models (modified LMs) and fuse them as done in Wortsman to improve accuracy without increasing inference time (Title, Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time).
Regarding claim 8, Shen in view of Wortsman teaches the system of claim 1 and the second modified dataset is created in accordance with at least one requirement of the LM, wherein the second modified dataset is utilized to train the LM (Par. 0093, The LM controller 407 includes a pre-processing module 413 and a language modelling (LM) module 415. The pre-processing module is configured to process inputs to prepare them for inputting into the language model (as described above). The LM module is configured to perform prediction based on the processed input. The LM controller may be configured to train the language model of the LM module in accordance with the training methods described herein).
Regarding claims 11 and 16, the claims teach essentially the same limitations as claim 1 with the addition of a non-transitory computer-readable medium which is taught by Shen (Claim 19, a non-transitory computer readable medium). As such, the claims are rejected for at least the same reasons.
Regarding claim 12, Shen in view Wortsman teaches the method of claim 11, and Shen teaches further comprising applying the first modified dataset to the LM to train the LM and create a first modified LM (Par. 0093, The LM controller 407 includes a pre-processing module 413 and a language modelling (LM) module 415. The pre-processing module is configured to process inputs to prepare them for inputting into the language model (as described above). The LM module is configured to perform prediction based on the processed input. The LM controller may be configured to train the language model of the LM module in accordance with the training methods described herein).
Regarding claim 17, Shen in view of Wortsman teaches the computer program product of claim 16, and Shen teaches wherein the operations further comprise: applying the first modified dataset to the LM to train the LM and create a first modified LM (Par. 0093, The LM controller 407 includes a pre-processing module 413 and a language modelling (LM) module 415. The pre-processing module is configured to process inputs to prepare them for inputting into the language model (as described above). The LM module is configured to perform prediction based on the processed input. The LM controller may be configured to train the language model of the LM module in accordance with the training methods described herein).
Claim(s) 4, 7, 13, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claims 1, 11, and 16 above, and further in view of Garg (US20190114360A1) and Deng (TURL: Table Understanding through Representation Learning).
Regarding claim 4, Shen in view of Wortsman teaches the system of claim 1 but fails to teach formatting the first original dataset in a tabular format to create the first modified dataset; identifying a first column of data in the first modified dataset as comprising input data, wherein the input data is to be applied to the LM; and identifying a second column of data in the first modified dataset as comprising output data comparable to data output from the LM.
Garg teaches formatting the first original dataset in a tabular format to create the first modified dataset (Par. 0020, Then, the canonical output generator 64 takes the standardized labels output by the classifier 63 and converts the standardized labels into a canonical data set, which is then stored in the databases 40. According to an embodiment, the canonical data set may be associated with JavaScript Object Notation (“JSON”). Other output formats can be supported by system as well, e.g., Microsoft Excel files, extensible markup language (XML), comma-separated values (CSV), etc.). Shen and Garg are analogous to the claimed invention because they are in the field of preparing data for further processing by natural language processing tools or language models. Therefore, it would have been for one of ordinary skill before the effective filling data of the claimed invention to have added the process of transforming data into a canonical structured format as done in Garg to Shen because some natural language processing tools require structured data (Par. 0003-4, And natural language processing tools are used to analyze text in context and extract desired information. Such digital tools generally require that the data being processed is structured, e.g., text stored in a recognized data structure, such as data organized in columns and rows for transactional system data or Microsoft Excel files).
The combination of Shen and Garg fails to teach identifying a first column of data in the first modified dataset as comprising input data, wherein the input data is to be applied to the LM; and identifying a second column of data in the first modified dataset as comprising output data comparable to data output from the LM.
Deng teaches identifying a first column of data in the first modified dataset as comprising input data, wherein the input data is to be applied to the LM; and identifying a second column of data in the first modified dataset as comprising output data comparable to data output from the LM (Table 2, table 2 displays splitting the data set into training and testing sets, columns from data tables are identified as training or testing columns).
Deng is analogous to the claimed invention because it’s in the field of processing tabular data
for use by language models. Therefore, it would have been obvious to one of ordinary skill before the effective filling data of the claimed invention to have used the method described in Deng along with the combination of Shen and Garg to capture the semantics and knowledge in large-scale unlabeled data
(Abs, and present a new Masked Entity Recovery (MER) objective for pre-training to capture the
semantics and knowledge in large-scale unlabeled data).
Regarding claim 7, Shen in view of Wortsman teaches the system of claim 1 but fails to teach identifying a first column of data in the first original dataset as comprising input data, identifying a second column of data in the first original dataset; comparing the content of the first column of data with the content of the second column of data to determine whether: the first column of data or the second column of data comprises input data; and the first column of data or the second column of data comprises output data.
Garg teaches identifying a first column of data in the first original dataset as comprising input data, identifying a second column of data in the first original dataset (Par. 0020, Then, the canonical output generator 64 takes the standardized labels output by the classifier 63 and converts the standardized labels into a canonical data set, which is then stored in the databases 40. According to an embodiment, the canonical data set may be associated with JavaScript Object Notation (“JSON”). Other output formats can be supported by system as well, e.g., Microsoft Excel files, extensible markup language (XML), comma-separated values (CSV), etc., turns the data into tables which requires identifying columns).
The combination of Shen and Garg fails to teach comparing the content of the first column of data with the content of the second column of data to determine whether: the first column of data or the second column of data comprises input data; and the first column of data or the second column of data comprises output data.
Deng teaches comparing the content of the first column of data with the content of the second column of data to determine whether: the first column of data or the second column of data comprises input data; and the first column of data or the second column of data comprises output data (Table 2, table 2 displays splitting the data set into training and testing sets, columns from data tables are identified as training or testing columns in an unsupervised manner).
Regarding claims 13 and 18, the claims teach essentially the same limitations as claim 4 and as such are rejected for at least the same reasons.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claim 1 above, and further in view of Fan (US20260050625A1).
Regarding claim 5, Shen in view of Wortsman teaches the system of claim 1 but fails to teach identifying a first collection of data in the first original dataset, wherein the first collection of data has a first language format; and converting the first collection of data to a second language format, wherein the second language format is a language required to train the LM.
Fan teaches identifying a first collection of data in the first original dataset, wherein the first collection of data has a first language format; and converting the first collection of data to a second language format, wherein the second language format is a language required to train the LM (Par. 0036, In some implementations, the pre-processor may be configured to remove anomaly text or formatted templates from the text data; process the text data according to a set of natural language processes (NLPs); and/or in response to a first portion of the text data being in a different language in comparison to a second portion of the text data, translate the first portion of the text data into a language being same as the second portion of the text data).
Shen and Fan are analogous to the claimed invention because they are both in the field of
endeavor of preparing/preprocessing data for model training. Therefore, it would have been obvious to
one of ordinary skill in the art before the effective filling date of the claimed invention to have added the
translating process of Fan into the pre-processing system of Shen to remove anomalies in text data (Par.
0036, In some implementations, the pre-processor may be configured to remove anomaly text or formatted templates from the text data; process the text data according to a set of natural language
processes (NLPs); and/or in response to a first portion of the text data being in a different language in
comparison to a second portion of the text data, translate the first portion of the text data into a
language being same as the second portion of the text data).
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claim 1 above, and further in view of Garg (US20190114360A1).
Regarding claim 6, Shen in view of Wortsman teaches the system of claim 1 but fails to teach identifying a base data format, wherein the base data format has a structure required for application of data to train the LM; applying the base data format to the first original dataset; and formatting the first original dataset to comply with the base data format.
Garg teaches identifying a base data format, wherein the base data format has a structure required for application of data to train the LM; applying the base data format to the first original dataset; and formatting the first original dataset to comply with the base data format (Par. 0020, Then, the canonical output generator 64 takes the standardized labels output by the classifier 63 and converts the standardized labels into a canonical data set, which is then stored in the databases 40. According to an embodiment, the canonical data set may be associated with JavaScript Object Notation (“JSON”). Other output formats can be supported by system as well, e.g., Microsoft Excel files, extensible markup language (XML), comma-separated values (CSV), etc.).
Garg and Shen are analogous to the claimed invention because they are all in the field of
endeavor of preparing data for model training. Therefore, it would have been obvious to one of ordinary
skill in the art before the effective filling data of the claimed invention to have added the process of transforming data into a canonical structured format as done in Garg to Shen because some natural
language processing tools require data in a specific format (Par. 0003-4, And natural language
processing tools are used to analyze text in context and extract desired information. Such digital tools
generally require that the data being processed is structured, e.g., text stored in a recognized data
structure, such as data organized in columns and rows for transactional system data or Microsoft Excel
files).
Claim(s) 9, 15, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claims 1, 11, and 16 above, and further in view of Walters (US20210073649A1).
Regarding claim 9, Shen in view of Wortsman teaches the system of claim 1 but fails to teach generating a first modified dataset from the first original dataset, wherein the first modified dataset comprises first data from a first column of data in the first original dataset with second data from a second column of data in the first original dataset; and generating a second modified dataset from the first original dataset, wherein the second modified dataset comprises the first data from the first column of data in the first original dataset with third data from a third column of data in the first original dataset.
Walters teaches generating a first modified dataset from the first original dataset, wherein the first modified dataset comprises first data from a first column of data in the first original dataset with second data from a second column of data in the first original dataset; and generating a second modified dataset from the first original dataset, wherein the second modified dataset comprises the first data from the first column of data in the first original dataset with third data from a third column of data in the first original dataset (Par. 0024, In some embodiments, one or more of the segments may be modified, for example, by dropping one or more columns of data, formatting one or more columns of data, and the like. Doing so may produce varying segments of training data 107, e.g., where a first segment has had a column dropped, a second segment has had a column formatted, a third segment has had one column dropped and one column formatted, and a fourth segment has not been modified).
Walters is analogous to the claimed invention because it is in the field of endeavor of data engineering by or for machine learning models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling data of the claimed invention to have added the data augmentation process done in Walters to the pre-processing in Shen to create additional/varying training samples to prevent overfitting (Par. 0024, Doing so may produce varying segments of training data 107, e.g., where a first segment has had a column dropped, a second segment has had a column formatted, a third segment has had one column dropped and one column formatted, and a fourth segment has not been modified).
Regarding claims 15 and 20, the claims teach essentially the same limitations as claim 9 and are rejected for at least the same reasons.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claim 1 above, and further in view of Wei (Tips and Tricks for Webly-Supervised Fine-Grained Recognition: Learning from the WebFG 2020 Challenge).
Regarding 10, Shen in view of Wortsman teaches the system of claim 1 but fails to teach analyzing a first original dataset; determining whether at least a portion of the first original dataset is corrupted data, wherein the corrupted data is data that cannot be opened; discarding a first portion of the first original dataset comprising corrupted data; and retaining a second portion of the first original dataset, wherein the second portion of the first original dataset comprises data for implementation in training the LM.
Wei teaches analyzing a first original dataset; determining whether at least a portion of the first original dataset is corrupted data, wherein the corrupted data is data that cannot be opened; discarding a first portion of the first original dataset comprising corrupted data; and retaining a second portion of the first original dataset, wherein the second portion of the first original dataset comprises data for implementation in training the LM (Section 2, Since our training images are directly crawled from the web, some broken images might be included. To remove these broken images, we employ the Python library, i.e., Pillow, to check each collected image and subsequently convert them to the RGB mode. Images that cannot be opened by Pillow or cannot be converted to RGB will be regarded as broken ones and get deleted).
Wei is analogous to the claimed invention because it is in the field of endeavor of pre-processing data for training on models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used the method of Wei alongside the combination of Shen and Wortsman to remove corrupted data like broken images.
Claim(s) 3 is rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Wortsman as applied to claim 1 above, and further in view of Wei and Mimura (US20250363412A1).
Regarding claim 3, Shen in view of Wortsman teaches the system of claim but fails to teach identifying a license associated with the first original dataset; determining a scope of the license; and in the event of the license does not permit use of the first original dataset to train the LM or that any portion of the LM is corrupted data and cannot be opened, rejecting implementation of the first original dataset.
Mimura teaches identifying a license associated with the first original dataset; determining a scope of the license; and in the event of the license does not permit use of the first original dataset to train the LM rejecting implementation of the first original dataset (Par. 0052, To be more specific, upon acquiring data identification information indicating the expiration date of the training dataset, such as when the license M expires, the data identifier 110 refers to the detailed information of the training dataset DB 20, and specifies the training datasets #001, #004, #006, and #008 having “license M” as the license information. The identified training data #001, #004, #006, and #008 become unusable after the expiration date……In this case, the processor 130 may retrain the machine learning model with the training data set from which the training data #001, #004, #006, and #008 have been deleted).
Mimura is analogous to the claimed invention because it is centered around managing and manipulating data to train machine learning models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling data of the claimed invention to have added the method of license verification found in Mimura into the data pre-processing in Shen to avoid legal or ethical issues (Par. 0023, Alternatively, issues from a legal and/or ethical point of view may be considered for the training data. As an example, for example, there may be a case where it is found after training of a machine learning model that training data for which commercial use is not permitted or licensed).
The combination of Shen, Wortsman, and Mimura fail to teach or that any portion of the LM is corrupted data and cannot be opened.
Wei teaches or that any portion of the LM is corrupted data and cannot be opened (Section 2, Since our training images are directly crawled from the web, some broken images might be included. To remove these broken images, we employ the Python library, i.e., Pillow, to check each collected image and subsequently convert them to the RGB mode. Images that cannot be opened by Pillow or cannot be converted to RGB will be regarded as broken ones and get deleted). The rationale for combination is described in the rejection for claim 10 above.
Response to Arguments
Applicant is reminded that all business is required to be done in writing (see MPEP 713.04). Nevertheless, this response covers the arguments presented in the response and during the interview prior to the filing. Applicant’s arguments, see Pg.8-11, filed 08/11/26, with respect to the objection, 35 U.S.C 112(b), and 35 U.S.C 101 have been fully considered and are persuasive. The objection of claim 8, 112(b) rejections of claims 1-10, 112(b) rejections of claims 3,8,14,19, and 101 rejections of claims 1-7,9-13,15-18,20 have been withdrawn.
Regarding the 35 U.S.C 102(a)(1) rejections, the applicant’s arguments are persuasive, given Shen no longer teaches every element of claims 1, 11, and 16; however, Shen in Combination with Wortsman teaches every element of claims 1, 11, and 16 and therefore new grounds for rejection of claims 1, 11, 12, 16, and 17 are made under 35 U.S.C 103 by Shen in view of Wortsman.
Regarding the 35 U.S.C 103 rejection of claim 3, Applicant’s arguments with respect to claim 3 have been considered but are moot because the new ground of rejection relies on Wei, which teaches the added portion discussing corrupted data that cannot be opened. Additionally, the applicant argues because claim 1 is no longer anticipated by Shen, the U.S.C 103 rejection for claim 3 Is invalid. Even though Shen no longer anticipates claim 1 alone, Shen in view of Wortsman teaches claim 1 and as such in combination with Mimura and Wei teaches claim 3.
Regarding the 35 U.S.C 103 rejections of claims 4, 7, 13, 18, the applicant argues because claims 1, 11, and 16 are no longer anticipated by Shen, the U.S.C 103 rejection for claims 4, 7, 13, and 18 Is invalid. Even though Shen no longer anticipates claims 1, 11, and 16 alone, Shen in view of Wortsman teaches claims 1, 11, and 16 and as such in combination with Garg and Deng teaches claims 4, 7, 13, and 18.
Regarding the 35 U.S.C 103 rejection of claim 5, the applicant argues because claim 1 is no longer anticipated by Shen, the U.S.C 103 rejection for claim 5 Is invalid. Even though Shen no longer anticipates claim 1 alone, Shen in view of Wortsman teaches claim 1 and as such in combination with Fan teaches claim 5.
Regarding the 35 U.S.C 103 rejection of claim 6, the applicant argues because claim 1 is no longer anticipated by Shen, the U.S.C 103 rejection for claim 6 Is invalid. Even though Shen no longer anticipates claim 1 alone, Shen in view of Wortsman teaches claim 1 and as such in combination with Garg teaches claim 6
Regarding the 35 U.S.C 103 rejection of claims 8, 14, 19, Applicant's arguments have been fully considered but they are not persuasive. The examiner believes Shen in view of Wortsman does teach the limitations of claim 8 (claims 14 and 19 are canceled).
Regarding the 35 U.S.C 103 rejection of claims 9, 15, 20, the applicant argues because claims 1, 11, and 16 are no longer anticipated by Shen the U.S.C 103 rejection for claims 9, 15, and 20 Is invalid. Even though Shen no longer anticipates claims 1, 11, and 16 alone, Shen in view of Wortsman teaches claims 1, 11, and 16 and as such in combination with Walters teaches claims 9, 15, 20.
Regarding the 35 U.S.C 103 rejection of claim 10, Applicant’s arguments with respect to claim 10 have been considered but are moot because the new ground of rejection relies on Wei, which teaches the added portion discussing corrupted data that cannot be opened. Additionally, the applicant argues because claim 1 is no longer anticipated by Shen the U.S.C 103 rejection for claim 10 Is invalid. Even though Shen no longer anticipates claim 1 alone, Shen in view of Wortsman teaches claim 1 and as such in combination with Wei teaches claim 10.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/NATNAEL A ASEGDEW/ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
