Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-11, 13, 15-18, and 20 are pending in this application. Claims 1, 11, 13, 18, and 20 are amended and claims 12, 14, and 19 are canceled by applicant’s amendment filed 17 December 2024.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11, 13, 15-18, and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1, it recites “the pre-queue workflow operating on the first set of input data to produce the second set of input data, the post-queue workflow operating on the second set of input data to produce a first set of output data.” The recited pre-queue workflow and post-queue workflow simply select which data from a set of input data are to be included in a set of output data. This is a mental process that evaluates data and judges which data to include.
This judicial exception is not integrated into a practical application because the only thing produced by the claim is a set of output data, which is an abstract idea; nothing practical is performed using the output data. The claims does not include additional elements that are sufficient to amount to significantly more than the judicial exception because all of the additional elements are either generic computing components or mere data gathering. The elements of a processor, a user device, a user interface, an input data source, and an application are all generic computing components, so they do not render the claim significantly more than an abstract idea. A queue is also a generic computing component, and creating a queue is a generic computer operation recited at a high degree of generality, so it too does not render the claims more than an abstract idea. The steps of “receiving . . . a collector identifier and information identifying an input data source producing a first set of input data,” “receiving . . . information identifying at least one of a pre-queue workflow and a post-queue workflow . . . and information identifying a data destination for the first set of output data,” and “outputting, by the processor to the data destination, the first set of output data for use in an application” are all mere data gathering, which is insignificant extra-solution activity and therefore does not render the claim more than an abstract idea.
Regarding Claim 2, it recites additional details about input data, which is an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 3, it recites additional details about input data, which is an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 4, it recites “wherein the pre-queue workflow applies at least a first threshold to attributes of the first set of input data.” Applying a threshold is a mental process that evaluates input data and judges whether it is higher or lower than a threshold value. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 5, it recites additional details about input data, which is an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 6, it recites additional details about input data, merely detailing what the data values represent; all of the forms of data are an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 7, it recites “wherein the pre-queue workflow applies a threshold to each set of input data.” Applying a threshold is a mental process that evaluates input data and judges whether it is higher or lower than a threshold value. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 8, it recites further details about the threshold. This is merely information about data, which is an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 9, it recites “wherein the post-queue workflow asynchronously operates on the second set of input data to produce the first set of output data.” Operating on data to produce a set of data is a mental process of judgement or evaluation that selects data from one set of data to produce a second set of data. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 10, it recites “wherein the post-queue workflow is a threshold model.” A “threshold model” simply compares data to a threshold, which is a mental process of judgement or evaluation as detailed above. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 11, it recites elements substantially similar to those of claim 1, so it recites an abstract idea without significantly more for the same reasons. The only additional element compared to claim 1 is “a network interface communication device.” This is a generic computing component recited at a high degree of generality, so it does not render the claim significantly more than an abstract idea.
Regarding Claim 13, it recites “wherein the pre-queue workflow is configured to select a random sample of data from the data source of the input data to be added to the queue, wherein the input data object is selected to be added to the queue.” Selecting a random sample of data is a mental process of judgement or evaluation that arbitrarily decides which data elements to select. Adding data to a queue is mere data gathering, which is insignificant extra-solution activity. The claim merely adds data to a queue, so it does not recite a practical application.
Regarding Claim 15, it recites further details about data, which is an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 16, it recites additional details about input data, merely detailing what the data values represent; all of the forms of data are an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 17, it recites additional details about input data, merely detailing what the data values represent; all of the forms of data are an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 18, it recites “wherein the post-queue workflow asynchronously operates on the input data object in the queue.” The post-queue workflow is a mental process that simply selects data, as described above. Specifying that the mental process is asynchronous does not render it more than a mental process. The claim does not recite a practical application and does not include any additional elements.
Regarding Claim 20, it recites “wherein the output data sink is a machine learning application.” A machine learning application is a generic computing component recited at a high degree of generality, so it does not render the claim significantly more than an abstract idea. The claim does not recite a practical application and does not include any additional elements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4-11, 15-18, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hendrycks, Dan, et al. (“Natural adversarial examples,” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021; hereinafter “Hendrycks.” Papers from CVPR Conference 2021 were publicly available on 11 June 2021, as shown on their website, https://cvpr2021.thecvf.com/ {accessed by the examiner 10 June 2026}).
Regarding Claim 1, Hendrycks teaches a computer implemented method to establish a data collector (Abstract and section 1—a method establishes a data collector to collect a second dataset to improve the training of a machine learning model), the method comprising:
receiving, by a processor from a user device operated by a user utilizing a user interface, a collector identifier and information identifying an input data source producing a first set of input data (section 1—an adversarial filtering method is received as a collector identifier that identifies a collection method, and ImageNet class images are downloaded from an identified as an input data source such as iNaturalist, Flickr, and DuckDuckGo);
creating, by the processor, a collector queue with the collector identifier to queue a second set of input data from the input data source (section 3.1—ImageNet-O is a second set of input data, which is generated by selecting samples from data of the downloaded ImageNet class images that have been processed by classifiers such as ResNet-50 {the first set of input data} that are classified with low confidence);
receiving, by the processor from the user device, information identifying at least one of a pre-queue workflow and a post-queue workflow, the pre-queue workflow operating on the first set of input data to produce the second set of input data, the post-queue workflow operating on the second set of input data to produce a first set of output data, and information identifying a data destination for the first set of output data (section 3.1—a pre-queue workflow is specified that classifies images in the first set of input data {the classified downloaded images} and removes images that are classified with high confidence. The post-queue workflow manually selects images from the second set of input data to ensure that the output data {ImageNet-A images} are valid, single-class, and high quality. Note that according to MPEP 2144.04 III, “providing an automatic or mechanical means to replace a manual activity which accomplished the same result is not sufficient to distinguish over the prior art”); and
outputting, by the processor to the data destination, the first set of output data for use in an application (sections 1 and 3.1—the output data set {ImageNet-A} is output for training a machine learning model).
Regarding Claim 11, Hendrycks teaches a computing system (section 4—experiments that train and test a machine learning system imply a computing system) comprising:
a network interface communication device configured to receive an input data object, the input data object received from a data source (section 3.1—downloading a dataset inherently requires a network interface communication device); and
a processor operably connected to the communication device (sections 3.1 and 4) and configured to:
identify at least a first collector associated with the data source (section 1—an adversarial filtering method is identified as a collector, and an ImageNet class of images are downloaded from an identified input data source such as iNaturalist, Flickr, and DuckDuckGo);
determine, based on a pre-queue workflow applying a first threshold to attributes of the input data object, to add the input data object to a queue of the at least first collector (section 3.1—a pre-queue workflow is specified that classifies images in the first set of input data {downloaded images} and removes images that are classified with high confidence, i.e. a confidence that exceeds a threshold);
add the input data object to the queue of the at least first collector (section 3.1—the first collector collects the selected images as the ImageNet-O dataset);
determine, based on a post-queue workflow applying a second threshold to attributes of the input data object, to pass the input data object from the queue to an output data sink (section 3.1—the post-queue workflow manually selects images from the second set of input data to ensure that the output data {ImageNet-A images} are valid, single-class, and high quality, thus determining a set of second thresholds for data to output to an output data sink. Note that according to MPEP 2144.04 III, “providing an automatic or mechanical means to replace a manual activity which accomplished the same result is not sufficient to distinguish over the prior art”); and
transmit the input data object to the output data sink (sections 1 and 3.1—the output data set {ImageNet-A} is output for training a machine learning model as an output data sink).
Regarding Claim 2, Hendrycks teaches wherein the first set of input data is the same as the second set of input data (section 3.1—the selection of images to include in the second set of data is made using confidence scores of a classifier, so if all images in the first set of input data are classified with low confidence, then the second set of input data will be the entire first set of input data).
Regarding Claim 4, Hendrycks teaches wherein the pre-queue workflow applies at least a first threshold to attributes of the first set of input data (section 3.1 {ImageNet-A Data Aggregation subheading}—a low-confidence threshold, such as 15%, is a first threshold applied to the first set of input data).
Regarding Claims 5 and 15, Hendrycks teaches wherein the input data source is a prediction stream of a machine learning model (section 3.1).
Regarding Claims 6 and 16, Hendrycks teaches wherein each set of input data from the prediction stream includes an input, at least a first predicted concept associated with the input, and at least a first confidence score associated with the at least first predicted concept (section 3.1—the input data stream includes, images, classifications, and a confidence score).
Regarding Claim 7, Hendrycks teaches wherein the pre-queue workflow applies a threshold to each set of input data (section 3.1).
Regarding Claims 8 and 17, Hendrycks teaches wherein the threshold is a threshold associated with the at least first confidence score associated with the at least first predicted concept (section 3.1, particularly the “ImageNet-A Data Aggregation” subheading).
Regarding Claims 9 and 18, Hendrycks teaches wherein the post-queue workflow asynchronously operates on the second set of input data to produce the first set of output data (section 3.1—the post-queue workflow is manual selection that is performed asynchronously to the pre-queue workflow. Note that according to MPEP 2144.04 III, “providing an automatic or mechanical means to replace a manual activity which accomplished the same result is not sufficient to distinguish over the prior art”).
Regarding Claim 10, Hendrycks teaches wherein the post-queue workflow is a threshold model (section 3.1—the judgement of images as being valid, single-class, and high quality is a model that applies a threshold to the second set of input data).
Regarding Claim 20, Hendrycks teaches wherein the output data sink is a machine learning application (sections 3.1 and 4).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Hendrycks, as applied to claims 1 and 11, above, in view of Cai et al. (U.S. 2020/0342265, hereinafter “Cai”).
Regarding Claim 3, Hendrycks does not specifically teach wherein the second set of input data is a random sample of the first set of input data. However, Cai teaches a second set of input data that is a random sample of a first set of input data (¶ [0024]).
All of the claimed elements were known in Hendrycks and Cai and could have been combined by known methods with no change in their respective functions. It therefore would have been obvious to a person of ordinary skill in the art at the time of filing of the applicant’s invention to combine the random sampling of Cai with the sets of input data of Hendrycks to yield the predictable result of wherein the second set of input data is a random sample of the first set of input data. One would be motivated to make this combination for the purpose of optimizing training of a machine learning model when a dataset is imbalanced (Cai, ¶ [0005]).
Regarding Claim 13, Hendrycks/Cai teaches wherein the pre-queue workflow is configured to select a random sample of data from the data source of the input data to be added to the queue, wherein the input data object is selected to be added to the queue (Cai, ¶ [0024]).
Response to Arguments
Applicant’s arguments filed 17 December 2024 with respect to the rejections under 35 U.S.C. 101 have been fully considered but they are not persuasive. The examiner has re-written the rejections above to better describe why the claims recite an abstract idea without significantly more. The applicant asserts that the claims are integrated into a practical application. Upon consideration, the examiner finds that there are no practical applications recited by the claims. Claim 1, for example, only produces a set of output data in the form of a collected dataset. The output data may be “for use in an application,” but nothing practical is performed using the output data. The remaining claims similarly lack a practical application because they too do not perform anything practical with the output data.
The applicant also asserts that the claims as a whole recite more than an abstract idea. Upon consideration, the examiner disagrees. Claim 1 as a whole, for example, recites a process of selecting data to include in a dataset. The selection steps can all be performed in the human mind as processes of judgement or evaluation—a human can consider each data element and decide whether or not to include it in the second set of input data and the first set of output data. As detailed above, the remaining elements in the claim are all generic computing components or input/output operations, which are forms of mere data gathering that are considered insignificant extra-solution activity. These elements therefore do not render the claim significantly more than an abstract idea. The remining claims comprise additional mental processes, generic computing components, and mere data gathering, as detailed above.
Applicant’s arguments with respect to the rejections under 35 U.S.C. 103 of claims 1-11, 13, 15-18, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The amendments change the scope of the claimed invention. Although Piechowicz (U.S. 2018/0004835) in view of Owen (U.S. 2022/0350814) does not teach all of the amended limitations, new prior art reference Hendrycks teaches all of the limitations of claims 1-2, 4-11, 15-18, and 20, as detailed above. Cai, in combination with Hendrycks, teaches claims 3 and 13, as also detailed above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. This art includes:
Delany, Sarah Jane, et al. (“Generating estimates of classification confidence for a case-based spam filter,” International conference on case-based reasoning. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005) teaches a spam filter that uses multiple classifiers and aggregates confidence scores to filter spam emails
Zhang, Jielun, et al. (“Autonomous unknown-application filtering and labeling for dl-based traffic classifier update,” IEEE INFOCOM 2020-IEEE conference on computer communications. IEEE, 2020) teaches a classifier that identifies and filters unknown types of communication packets and adds them to a training dataset for training a machine learning system based on a confidence score from the classifier
Ali, Syed Muhammad Fawad, Johannes Mey, and Maik Thiele (“Parallelizing user-defined functions in the ETL workflow using orchestration style sheets,” International Journal of Applied Mathematics and Computer Science 29.1 (2019): 69-79) teaches a system that automates parallel programming of user-defined ETL functions for filtering datasets
Qu et al. (U.S. 2022/0318672) teaches a defect classifier that removes unconfident samples from a dataset and uses the confident samples to train a classifier and an outlier detector
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAL W SCHNEE whose telephone number is (571) 270-1918. The examiner can normally be reached M-F 7:30 a.m. - 6:00 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at 303-297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAL SCHNEE/Primary Examiner, Art Unit 2129
Prosecution Insights