DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 3/28/2024. Claims 1-21 are pending and have been examined. Claims 1-21 are rejected.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because Fig. 7B includes the reference character 704a-3, which is not mentioned in the description (see, paragraphs 130-132 describing FIG. 7B).
Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In paragraph 0028, lines 6 and 8 contain a double period
In paragraph 0055, line 2, there seems to be a word missing after “other”
In paragraph 0088, line 21, “column” should read “columns”
In paragraph 0090, “ML” should be preceded by the article “an” instead of “a”
In paragraph 00114, “configure” should read “configured”
In paragraph 00120, line 8, “score” should be omitted
Reference character 704a-3 shown in Figure 7B is not found in the detailed description (see, paragraphs 130-132 describing FIG. 7B).
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: a processing entity operable to: <perform operations> in claims 8-14.
Regarding claims 8-14 and the above-noted three-prong test, the recited processing entity is a generic placeholder, operable to: <perform the operations specified in each of claims 8-14> is functional language, and there is no recitation in the claims of sufficient structure to perform the recited operations of claims 8-14.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows that the corresponding structure is not described in the specification for the 35 U.S.C. 112(f) limitations. Aside from merely repeating the claim language in paragraphs 17, 28, 128-130 and 133-134 of the specification, the high-level block diagram of FIG.11 depicts a processor 1110; however, it is unclear if such processor is the claimed processing entity. Thus, the specification does not provide descriptions of the structure of the processing entity.
As such, the specification either fails to describe the claimed processing entity as noted above, or, at best, describes the claimed processing entity by its functions without disclosing any specific structure performing the claimed functions. Thus, for these claim limitations, the written description fails to disclose what exactly the processing entity is.
Accordingly, for these claim limitations, the written description fails to disclose both an algorithm(s) and special-purpose computer hardware to perform the algorithm(s). For more information, see MPEP § 2181.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 8-14 are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement.
Independent claim 8 and dependent claims 9-14 each contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention.
In particular, and as previously noted, the claim limitation a processing entity in claims 8-14 invokes 35 U.S.C. 112(f).
However, as noted above, the written description of the current application fails to disclose the corresponding structure, material, or acts for performing each of the above-identified claimed functions and to clearly link the structure, material, or acts to the function. In particular, for each of the claimed functions, the written description fails to disclose both an algorithm(s) and special-purpose computer hardware to perform the algorithm. For more information, see MPEP § 2181.
Accordingly, claims 8-14 are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 8-14 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
As discussed above, the claim limitation processing entity in claims 8-14 invokes 35 U.S.C. 112(f).
However, as also discussed above with regard to the rejections of independent claim 8 and dependent claims 9-14 under 35 U.S.C. 112(a), the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. In particular, the specification fails to clearly link the structure, material, or acts to the function for the limitation processing entity in claims 8-14. As further noted above, there is insufficient disclosure in the specification of what exactly the processing entity is, as well as any algorithms and specific computer hardware for implementing it. As such, the above-noted limitations recited in claims 8-14 are indefinite. Therefore, claims 8-14 are indefinite and are rejected under 35 U.S.C. 112(b).
For the purposes of determining patent eligibility and comparison with the prior art, the examiner is interpreting the above-listed processing entity as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7 and 12-25 of copending Application No. 18/620,682 (hereinafter ‘682). Although the claims at issue are not identical, they are not patentably distinct from each other because all the limitations of claims 1-21 in the present application are covered by claims 1-7 and 12-25 of ‘682 (please see the table below).
Regarding claims 1-21, claims 1-7 and 12-25 of ‘682 teaches the claimed inventions as shown in the table below.
This is a provisional nonstatutory double patenting rejection since the claims directed to the same invention have not in fact been patented.
Instant app 18/620,720 (as filed 03/28/2024)
Co-pending reference app 18/620,682 (as filed 03/28/2024)
1. A method implemented by a computing apparatus for selecting features within a data set to use for outlier detection analysis; the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria; the method comprising, automatically by the computing apparatus:
- selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
- for each candidate feature, determining an information content metric for values of data entries within the criteria of the candidate feature; and
- selecting one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics.
1. A method implemented by a computing apparatus for identifying potential outlier data entries within a data set; the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria; the method comprising, automatically by the computing apparatus: …
2. The method of claim 1, wherein selecting the plurality of features of the data set to use within the outlier detection operations comprises:
- selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
- for each of the candidate features, determining a respective information content metric representative of all values of data entries within the criteria of the candidate features; and
- selecting candidate features to be the features of the data set to use within the outlier detection operations at least partially using the information content metrics of the candidate features.
2. The method of claim 1, wherein the selecting the features within the data set to use for outlier detection analysis further comprises: prior to selecting the one or more of the candidate features, determining one or more pairwise correlations between the candidate features; and, if one of the correlations, between two of the candidate features, is above a predetermined threshold, inhibiting using both of the candidate features associated with the one of the correlations.
3. The method of claim 2, wherein selecting the plurality of features of the data set to use within the outlier detection operations further comprises: determining correlation between the candidate features prior to selecting the candidate features; and, if a correlation between two of the candidate features is above a predetermined threshold, inhibiting using both of the correlated candidate features.
3. The method of claim 1, wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
4. The method of claim 2, wherein determining the information content metric for values of data entries within the criteria of the candidate feature comprises determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
4. The method of claim 1, wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises using entropy calculations on the values of data entries within the criteria.
5. The method of claim 2, wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises using entropy calculations on the values of data entries within the criteria.
5. The method of claim 1 further comprising:
- selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features;
- generating an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and
- combining the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
1. A method …
- selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run using values of the data set corresponding to one of the selected features as input;
- generating, for each one of a set of data items and for each operation of the plurality of selected outlier detection operations, a respective outlier characteristic metric for said one of the set of data items, the generating using said operation applied to said one of the set of data items, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and
- for each one of the set of data items, combining the plurality of outlier characteristic metrics for said one of the set of data items to generate an ensemble outlier metric for said one of the set of data items.
6. The method according to claim 5, wherein the selecting the plurality of outlier detection operations comprises: determining which of a plurality of candidate outlier detection algorithms can use the selected features; running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
6. The method according to claim 1, wherein the selecting the plurality of outlier detection operations comprises: determining which of a plurality of candidate outlier detection algorithms can use the selected features; running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
7. The method according to claim 6, wherein the selecting a plurality of outlier detection operations further comprises: determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
7. The method according to claim 6, wherein the selecting the plurality of outlier detection operations further comprises: determining a diversity metric for each of the candidate algorithm results relative to some or all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to diversity metrics for other candidate algorithm results.
8. A computing apparatus for selecting features within a data set to use for outlier detection analysis, the computing apparatus comprising:
- a processing entity operable to: receive a data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria; select a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria; for each candidate feature, determine an information content metric for values of data entries within the criteria of the candidate feature; and select one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics.
12. A computing apparatus for identifying potential outlier data entries within a data set, the computing apparatus comprising: …
13. The computing apparatus of claim 12, wherein to select the plurality of features of the data set to use within the outlier detection operations, the processing entity is operable to: select a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria; determine an information content metric for values of data entries within the criteria of each of the candidate features; and select candidate features to be the features of the data set to use within the outlier detection operations at least partially using the information content metrics of the candidate features.
9. The computing apparatus of claim 8, wherein to select the features within the data set to use for outlier detection analysis, the processing entity is operable to: prior to selecting the one or more of the candidate features, determine one or more pairwise correlations between the candidate features; and, if one of the correlations, between two of the candidate features, is above a predetermined threshold, filter one of the candidate features associated with the one of the correlations.
14. The computing apparatus of claim 13, wherein to select the plurality of features of the data set to use within the outlier detection operations, the processing entity is further operable to: determine correlation between the candidate features prior to selecting the candidate features; and, if a correlation between two of the candidate features is above a predetermined threshold, filter one of the correlated candidate features.
10. The computing apparatus of claim 8, wherein to determine the information content metric for values of data entries within the criteria of the candidate feature, the processing entity is operable to: determine a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
15. The computing apparatus of claim 13, wherein to determine the information content metric for values of data entries within the criteria of the candidate feature, the processing entity is operable to: determine a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
11. The computing apparatus of claim 8, wherein to determine the information content metric for values of data entries within the criteria of the candidate feature, the processing entity is operable to: use entropy calculations on the values of data entries within the criteria.
16. The computing apparatus of claim 13, wherein to determine the information content metric for values of data entries within the criteria of the candidate feature, the processing entity is operable to: use entropy calculations on the values of data entries within the criteria.
12. The computing apparatus of claim 8, wherein the processing entity is further operable to: select a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features; generate an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and combine the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
12. A computing apparatus …
- a processing entity operable to: … select a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features; generate an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and combine the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
13. The computing apparatus of claim 12, wherein to select the plurality of outlier detection operations, the processing entity is operable to: determine which of a plurality of candidate outlier detection algorithms can use the selected features; run each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determine an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and select the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
17. The computing apparatus according to claim 12, wherein to select the plurality of outlier detection operations, the processing entity is operable to: determine which of a plurality of candidate outlier detection algorithms can use the selected features; run each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determine an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and select a plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
14. The computing apparatus of claim 13, wherein to select the plurality of outlier detection operations, the processing entity is further operable to: determine a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
18. The computing apparatus according to claim 17, wherein to select the plurality of outlier detection operations, the processing entity is further operable to: determine a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the selecting a plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
15. Non-transitory computer-readable media containing a program element executable by a computing system to perform a method for selecting features within a data set to use for outlier detection analysis; the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria; the method comprising:
- selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
- for each candidate feature, determining an information content metric for values of data entries within the criteria of the candidate feature; and
- selecting one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics.
19. Non-transitory computer-readable media containing a program element executable by a computing system to perform a method for identifying potential outlier data entries within a data set; the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria; the computer-readable media comprising:
- first program code for selecting a plurality of features of the data set to use within one or more outlier detection operations …
20. The non-transitory computer-readable media of claim 19, wherein the first program code comprises:
- fifth program code for selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
- sixth program code for determining an information content metric for values of data entries within the criteria of each of the candidate features; and
- seventh program code for selecting candidate features to be the features of the data set to use within the outlier detection operations at least partially using the information content metrics of the candidate features.
16. The non-transitory computer-readable media of claim 15, wherein the selecting the features within the data set to use for outlier detection analysis further comprises: prior to selecting the one or more of the candidate features, determining one or more pairwise correlations between the candidate features; and, if one of the correlations, between two of the candidate features, is above a predetermined threshold, inhibiting using both of the candidate features associated with the one of the correlations.
21. The non-transitory computer-readable media of claim 20, wherein the first program code further comprises: program code for determining correlation between the candidate features prior to selecting the candidate features; and program code for, if a correlation between two of the candidate features is above a predetermined threshold, inhibiting using both of the correlated candidate features.
17. The non-transitory computer-readable media of claim 15, wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
22. The non-transitory computer-readable media of claim 20, wherein the sixth program code comprises program code for determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
18. The non-transitory computer-readable media of claim 15, wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises using entropy calculations on the values of data entries within the criteria.
23. The non-transitory computer-readable media of claim 20, wherein the sixth program code comprises program code for using entropy calculations on the values of data entries within the criteria.
19. The non-transitory computer-readable media of claim 15, wherein the method further comprises:
- selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features;
- generating an outlier characteristic metric for each of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and
- combining the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
19. Non-transitory computer-readable media … the computer-readable media comprising:
- second program code for selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features;
- third program code for generating an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and
- fourth program code for combining the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
20. The non-transitory computer-readable media of claim 19, wherein the selecting the plurality of outlier detection operations comprises: determining which of a plurality of candidate outlier detection algorithms can use the selected features; running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
24. The non-transitory computer-readable media according to claim 19, wherein the second program code comprises: program code for determining which of a plurality of candidate outlier detection algorithms can use the selected features; program code for running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results; determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection; and program code for selecting a plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
21. The non-transitory computer-readable media of claim 20, wherein the selecting a plurality of outlier detection operations further comprises: determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
25. The non-transitory computer-readable media according to claim 24, wherein the second program code further comprises: program code for determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; wherein the second program code is operable to select a plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
The table above uses the underlined text to highlight the similarities between the two applications and claims within, while the non-underlined text signifies the difference between the claim sets. Upon review of the non-underlined text, a person of ordinary skill in the art would conclude that the scope of the claim inventions are obvious variants of each other. To further expand on the above table, each claim listed above is either directly from the reference patent and/or obvious as explained below. Claims not directly referenced in the explanation below are believed to be adequately explained by the comparison chart above.
Regarding independent claims 1, 8 and 15 of the instant application, the claim languages and scope are similar to one another. The examiner points out the similarities between claim 1 of this application and claims 1-2 of the co-pending application in this office action for clarity but the reasoning, motivation(s), and rationale apply for instant claims 8 and 15 as well.
Regarding independent claims 1, 8, and 15 of the instant application, these claims are taught by claims 1-2, 12-13, and 19-20, respectively, of the co-pending reference ‘682 application. The only difference in the claims is that the part of the preambles in the instant application claims that state selecting features within a data set to use for outlier detection analysis is a slightly reworded version of the recitation of selecting a plurality of features of the data set to use within one or more outlier detection operations in the reference ‘682 application. However, this is semantically the same. That is, instant claims 1, 8 and 15 are slightly reworded versions of claims 1-2, 12-13, and 19-20 of the reference application and claims 1-2, 12-13, and 19-20 of the reference application encompasses the same subject matter as instant claims 1, 8 and 15, respectively.
Regarding instant dependent claims 5, 12, and 19, they are taught by claims 1, 12, and 19, respectively, of the reference application ‘682. As shown in the table above, instant claims 5, 12, and 19 depend on claims 1, 8, and 15, respectively, which are taught by claims 1-2, 12-13, and 19-20 of reference application ‘682 respectively,.
Regarding instant dependent claims 6-7, 13-14, and 20-21, they are taught by claims 6-7, 17-18, and 24-25, respectively, of reference application ‘682. As also shown in the table above, instant claims 6-7, 13-14, and 20-21 depend on intervening claims 5, 12, and 19 respectively, which are taught by claims 1, 12, and 19, respectively, of reference application ‘682.
Regarding instant claims 15-21, they are taught by claims 19-25 of reference application ‘682. The only difference in the claims is that claims 15-21 of the instant application recite a program element executable by a “computing system to perform a method” while claims 19-25 of ‘682 further recite the program element as a collection of program code to carry out the same steps. That is, claims 15-21 of the instant application are slightly broader versions of claims 19-25 of reference application ‘682. Therefore, the recitations of claims 19-25 of reference application ‘682 encompass the same subject matter as claims 15-21 of the instant application.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 U.S.C. 101. because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding independent claims 1 and 15, these claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 1 is directed to a method, corresponding to a process, and claim 15 is directed to a non-transitory computer readable media, corresponding to an article of manufacture; both claims correspond to one of the statutory categories.
2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claims 1 and 15 recite, using respective similar language, the following limitations:
for selecting features within a data set
selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
for each candidate feature, determining an information content metric for values of data entries within the criteria of the candidate feature;
and selecting one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics.
The above limitations encompass selecting/choosing features from a data set based on a criteria (evaluation/judgement/opinion based on an observed data set and criteria), determining an information content metric for the values in the features (evaluation/judgement/opinion based on observed features, data values, and criteria), and then selecting the features to be used for outlier detection analysis (evaluation/judgement/opinion based on observed features and information content metrics). All these limitations can be performed in the human mind, or with the aid of pen and paper.
2A Prong 2 Analysis:
The preambles of claims 1 and 15 both recite the additional elements:
for selecting features within a data set to use for outlier detection analysis;
the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria.
These are insignificant extra-solution activities that are not integrated into the claims as a whole and do not add a meaningful limitation to the above-noted mental processes specified in these claims. That is, features within a data set to use for outlier detection analysis and the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria amounts to mere data gathering. See MPEP § 2106.05(g).
Claim 1 recites the additional elements: implemented by a computing apparatus …the method comprising, automatically by the computing apparatus: <performing the above-noted steps>.
Claim 15 similarly recites: Non-transitory computer-readable media containing a program element executable by a computing system to perform <the above-noted operations>.
These elements amount to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Receiving, communicating, forwarding and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)". Therefore, the recitations of a data set to use for outlier detection analysis and the data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria are the well-understood, routine, conventional activities receiving and transmitting data over a network and storing information in memory, as discussed in MPEP § 2106.05(d).
Also, mere instructions to apply the mental process electronically (implemented by a computing apparatus … the method comprising, automatically by the computing apparatus (claim 1) and Non-transitory computer-readable media containing a program element executable by a computing system (claim 15)) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for choosing features from a data set based on a criteria, determining/identifying an information content metric for the values in the features, and then selecting the features to be used for outlier detection analysis. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding independent claim 8, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to an apparatus, corresponding to an article of manufacture, which is one of the statutory categories.
Step 2A Prong 1 Analysis:
The claim is directed to an abstract idea. In particular, the claim recites mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claim 8 recites the following limitations:
selecting features within a data set … selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria;
for each candidate feature, determining an information content metric for values of data entries within the criteria of the candidate feature;
and selecting one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics.
The above limitations encompass choosing features from a data set based on a criteria (evaluation/judgement/opinion based on an observed data set and criteria), determining an information content metric for the values in the features (evaluation/judgement/opinion based on observed features, data values, and criteria), and then selecting the features to be used for outlier detection analysis (evaluation/judgement/opinion based on observed features and information content metrics). All these limitations can be performed in the human mind, or with the aid of pen and paper.
Step 2A Prong 2 Analysis:
Claim 8 recites the additional elements:
features within a data set to use for outlier detection analysis … and
receive a data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria.
These are insignificant extra-solution activities that are not integrated into the claim as a whole and do not add a meaningful limitation to the above-noted mental processes specified in this claim. That is, features within a data set to use for outlier detection analysis and receive a data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria amount to mere data gathering. See MPEP § 2106.05(g).
Claim 8 recites the additional element: the computing apparatus comprising: a processing entity1 operable to: <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Receiving, communicating, forwarding and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)". Therefore, the recitations of features within a data set to use for outlier detection analysis and receive a data set comprising a plurality of data entries, each of the data entries comprising a value for some or all of a plurality of criteria are the well-understood, routine, conventional activities receiving and transmitting data over a network and storing information in memory, as discussed in MPEP § 2106.05(d).
Also, mere instruction to apply the mental process electronically (the computing apparatus comprising: a processing entity operable to) does not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for choosing features from a data set based on a criteria, determining an information content metric for the values in the features, and then selecting the features to be used for outlier detection analysis. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 2, 9 and 16, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 2 is directed to a method as depending from claim 1, claim 9 is directed to a computing apparatus as depending from claim 8, and claim 16 is directed to non-transitory computer readable media as depending from claim 15; thus, the analysis for patent eligibilities of claims 1, 8, and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claims 2, 9 and 16 recite, using respective similar language, the following limitations:
prior to selecting the one or more of the candidate features, determining one or more pairwise correlations between the candidate features;
and, if one of the correlations, between two of the candidate features, is above a predetermined threshold, inhibiting using both of the candidate features associated with the one of the correlations.
The above limitations encompass determining pairwise correlations between the features (evaluation/judgement/opinion based on observed features) and then preventing the use of both features in the pair if their correlation is above a predetermined threshold (evaluation/judgement/opinion based on the observed features, threshold and correlation value). All these limitations are mental processes that can be performed in the human mind, or with the aid of pen and paper.
Step 2A Prong 2 Analysis:
Claims 2, 9 and 16 recite, using respective similar language, the additional element:
wherein the selecting the features within the data set to use for outlier detection analysis further comprises:
The additional element does not integrate the judicial exception into practical application as it is mere restatement of a portion of the parent claims.
Claim 9 recites the additional element: the processing entity2 is operable to <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere restatement of a portion of the parent claims and instruction to apply the mental process electronically (the processing entity is operable to) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for determining pairwise correlations between the features and then preventing the use of both features in the pair if their correlation is above a predetermined threshold. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 3, 10 and 17, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
Claim 3 is directed to a method as depending from claim 1, claim 10 is directed to a computing apparatus as depending from claim 8, and claim 17 is directed to non-transitory computer readable media as depending from claim 15; thus, the analysis for patent eligibilities of claims 1, 8, and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mathematical concepts (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations).
Claims 3, 10 and 17 recite, using respective similar language, the following limitations:
determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature - Under their BRI, in light of the specification, the determining limitation covers a mathematical concept - i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations – to determine/calculate “a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.”).
The above limitation encompasses determining a potential compression rate for the values in the features (mathematical concept using “the ratio between an uncompressed data size and a compressed data size” as disclosed in the specification – [0078]). This limitation is a mathematical concept using the ratio of two values.
Therefore, the claims are directed to an abstract idea – the mental processes of base claims 1, 8 and 15 combined with the above-noted mathematical concept.
Step 2A Prong 2 Analysis:
Claims 3, 10 and 17 recite, using respective similar language, the additional element:
wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises
The additional element as disclosed above does not integrate the judicial exception into practical application as it is mere restatement of a portion of the parent claims.
Claim 10 recites the additional element: the processing entity3 is operable to: <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere restatement of a portion of the parent claims and instruction to apply the mental process electronically (the processing entity is operable to) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for determining a potential compression rate for the values in the features. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 4, 11, and 18, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 4 is directed to a method as depending from claim 1, claim 11 is directed to a computing apparatus as depending from claim 8, and claim 18 is directed to non-transitory computer readable media as depending from claim 15; thus, the analysis for patent eligibilities of claims 1, 8, and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mathematical concepts.
Claims 4, 11 and 18 recite, using respective similar language, the following limitations:
using entropy calculations on the values of data entries within the criteria.
The above limitation encompasses performing entropy calculations on the values of the data entries (mathematical concept using the Shannon entropy formula as disclosed in the specification – [0078]). This limitation is a mathematical calculation done through the application of the Shannon entropy formula on the values of data entries within the criteria.
Step 2A Prong 2 Analysis:
Claims 4, 11 and 18 recite, using respective similar language, the additional element:
wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises
The additional element as disclosed above does not integrate the judicial exception into practical application as it is mere restatement of a portion of the parent claims.
Claim 11 recites the additional element:
the processing entity4 is operable to <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere restatement of a portion of the parent claims and instruction to apply the mental process electronically (the processing entity is operable to) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for performing entropy calculations on the values of the data entries. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 5, 12 and 19, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 5 is directed to a method as depending from claim 1, claim 12 is directed to a computing apparatus as depending from claim 8, and claim 19 is directed to non-transitory computer readable media as depending from claim 15; thus, the analysis for patent eligibilities of claims 1, 8 and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claims 5, 12 and 19 recite, using respective similar language, the following limitations:
selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features;
generating an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; and
combining the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items.
The above limitations encompass choosing outlier detection operations (evaluation/judgement/opinion based on an observed features and algorithms), generating an outlier characteristic metric for each set of data items (evaluation/judgement/opinion based on observed outlier detection operations and data entries), and then combining the outlier characteristic metrics to generate an ensemble outlier metric (evaluation/judgement/opinion based on observed outlier characteristic metrics and data entries). All these limitations can be performed in the human mind, or with the aid of pen and paper.
Step 2A Prong Two Analysis:
The judicial exceptions are not integrated into a practical application.
Claims 5 and 19 do not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Claim 12 recites the additional element:
the processing entity5 is further operable to <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere instruction to apply the mental process electronically (the processing entity is further operable to) does not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for choosing outlier detection operations, generating an outlier characteristic metric for each set of data items, and then combining the outlier characteristic metrics to generate an ensemble outlier metric. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 6, 13 and 20, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 6 is directed to a method as depending from claim 5, claim 13 is directed to a computing apparatus as depending from claim 12, and claim 20 is directed to non-transitory computer readable media as depending from claim 19; thus, the analysis for patent eligibilities of claims 5, 12, and 19, and of base claims 1, 8 and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claims 6, 13 and 20 recite, using respective similar language, the following limitations:
determining which of a plurality of candidate outlier detection algorithms can use the selected features;
run each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results;
determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection;
and selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
The above limitations encompass determining which outlier detection algorithms are compatible with the features (evaluation/judgement/opinion based on observed features and outlier detection algorithms), applying the outlier detection algorithms on compatible features to generate algorithm results (evaluation/judgement/opinion based on the observed features and outlier detection algorithms), determining a metric for the effectiveness of the algorithms in detecting outliers (evaluation/judgement/opinion based on the observed features, algorithms, and algorithm results), and selecting which of the algorithms to use along with the features (evaluation/judgement/opinion based on the observed features, algorithms, and effectiveness metrics). All these limitations are mental processes that can be performed in the human mind, or with the aid of pen and paper.
Step 2A Prong 2 Analysis:
Claims 6, 13 and 20 recite, using respective similar language, the additional element:
wherein the selecting the plurality of outlier detection operations comprises:
The additional element as disclosed above does not integrate the judicial exception into practical application as it is mere restatement of a portion of the parent claims.
Claim 13 recites the additional element:
the processing entity6 is operable to <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere restatement of a portion of the parent claims and instruction to apply the mental process electronically (the processing entity is operable to) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for determining which outlier detection algorithms are compatible with the features, applying the outlier detection algorithms on compatible features to generate algorithm results, determining a metric for the effectiveness of the algorithms in detecting outliers, and selecting which of the algorithms to use along with the features. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Regarding dependent claims 7, 14 and 21, the claims are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis:
Claim 7 is directed to a method as depending from claim 6, claim 14 is directed to a computing apparatus as depending from claim 13, and claim 21 is directed to non-transitory computer readable media as depending from claim 20; thus, the analysis for patent eligibilities of claims 6, 13 and 20, of intervening claims 5, 12 and 19, and of base claims 1, 8 and 15 are incorporated herein.
Step 2A Prong 1 Analysis:
The claims are directed to an abstract idea. In particular, the claims recite mental processes that are concepts performed in the human mind (including observation, evaluation, judgement, and opinion).
Claims 7, 14 and 21 recite, using respective similar language, the following limitations:
determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results;
wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
The above limitations encompass determining a diversity metric measuring the correlation between information in the algorithm results obtained in the steps of the parent claims (evaluation/judgement/opinion based on the observed features, algorithms, and algorithm results), and selecting which of the algorithms to use along with the features (evaluation/judgement/opinion based on the observed features, algorithms, and diversity metrics). All these limitations are mental processes that can be performed in the human mind, or with the aid of pen and paper.
Step 2A Prong 2 Analysis:
Claims 7, 14 and 21 recite, using respective similar language, the additional element:
wherein the selecting the plurality of outlier detection operations further comprises:
The additional element as disclosed above does not integrate the judicial exception into practical application as it is mere restatement of a portion of the parent claims.
Claim 14 recites the additional element:
the processing entity7 is further operable to <perform the above-noted operations>.
This element amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instruction to implement an abstract idea or other exception on a computer or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
The additional elements recited above alone or in combination do not integrate the judicial exception into practical application as they are mere restatement of a portion of the parent claims and generic computer functions that are implemented to perform the disclosed abstract idea above.
Step 2B Analysis:
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Viewing the additional elements of these claims as a combination does not add anything further than the individual elements.
The additional elements do not amount to significantly more than the judicial exception for the same reasons that they do not integrate the judicial exception into a practical application.
Also, mere restatement of a portion of the parent claims and instruction to apply the mental process electronically (i.e., the processing entity is further operable to as recited in claim 14) do not amount to significantly more than the judicial exception.
Accordingly, at Step 2B, the additional elements do not amount to significantly more than the judicial exception.
As an ordered whole, the claims are directed to a method for determining a diversity metric measuring the correlation between information in the algorithm results obtained in the steps of the parent claims and then selecting which of the algorithms to use along with the features. Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 4, 5, 8, 9, 11, 12, 15, 16, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Fu et al. (“REMIX: Automated Exploration for Interactive Outlier Detection”, 2017, hereinafter “Fu”) in view of Aggarwal et al (US 20170228432 B1, hereinafter “Aggarwal”).
With respect to claim 1, Fu discloses the invention as claimed including a method … the method comprising, automatically (see, e.g., page 2, section 1.1, the system presented in Fu, REMIX, is a “framework for automated outlier exploration”; FIG. 2 – a flowchart of a method performed by REMIX),
for selecting features within a data set to use for outlier detection analysis; the data set comprising a plurality of data entries (see, e.g., page 3, section 1.1.3, “the diversity of feature subspaces and algorithms explored by REMIX will result in different detectors marking a distinct set of data points [i.e., data entries] as outliers”; page 4, section 3.2, “An outlier detector Da,f is simply a combination of an algorithm a ∈ A and a feature subspace f of A” [i.e., REMIX selects features and then does an outlier exploration – corresponds to outlier detection analysis – using selected features and algorithms]),
each of the data entries comprising a value for some or all of a plurality of criteria8 (see, e.g., page 9, section 5.1, “these datasets span varied domains [i.e., criteria] such as flowers, breast cancers, heart diseases, …”; [i.e., the domains of the datasets are equivalent to the criteria recited in the claim under the BRI of criteria as a characteristic of a data set as a whole]);
selecting a plurality of candidate features of the data set, each of the candidate features comprising or being derived from one or more of the criteria (see, e.g., page 6, section 4.1, “we select Γ subspaces, where each subspace consists of features selected independently…”);
for each candidate feature, determining an information content metric for values of data entries within the criteria of the candidate feature (see, e.g., page 6, section 4.1, “The family of prioritized feature subspaces Fp is created as follows. The features in Fnr are first sorted according to their Laplacian scores … We add |Fnr| subspaces to F¬p, where the lth subspace is the set of top-features in Fnr ranked by their Laplacian scores”; page 5, algorithm 1, lines 14-18, the for loop is depicting this process of selecting features based on their calculated Laplacian scores; page 7, section 4.3, “These statistics include the Laplacian score …, and entropy which is a measure of the information content in the feature.” [i.e., Laplacian scores are used in algorithm 1 for the ranking, but can be substituted for entropy, corresponding to an information content metric]); and
selecting one or more of the candidate features to be the features within the data set to use for outlier detection analysis at least partially using the determined information content metrics9 (see, e.g., page 6, section 4.1, “The family of prioritized feature subspaces Fp is created as follows. The features in Fnr are first sorted according to their Laplacian scores … We add |Fnr| subspaces to F-p, where the lth subspace is the set of top-features in Fnr ranked by their Laplacian scores”; page 5, algorithm 1, lines 14-18, the for loop is depicting this process of selecting features based on their calculated Laplacian scores; page 7, section 4.3, “These statistics include the Laplacian score …, and entropy which is a measure of the information content in the feature.” [i.e., Laplacian scores are used in algorithm 1 for the ranking, but can be substituted for entropy, corresponding to an information content metric]).
Although Fu substantially discloses the claimed invention, Fu does not explicitly disclose a method implemented by a computing apparatus … by the computing apparatus: <perform the above -noted steps/operations>
However, in the same field, analogous art Aggarwal teaches a method implemented by a computing apparatus … by the computing apparatus (see, e.g., FIG. 4 – depicting “outlier detection server 402” [i.e., a server/computing apparatus]; paragraphs 56, “operations of the methods 200 or 300 may be performed by an outlier detection server 402” and 68, “a special-purpose computer or other particular machine, a processor, or other programmable data processing apparatus … execution of the instructions on the computer, processor, or other programmable data processing apparatus causes one or more functions or operations specified in the flow diagrams to be performed. … The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process.” [i.e., a method/process implemented by the server/computing apparatus]).
Fu and Aggarwal are analogous art because they are both directed to methods for outlier detection. (see, e.g., Fu, abstract; Aggarwal, abstract)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the disclosed method of Fu to incorporate the teachings of Aggarwal in order to have the method be “performed by an outlier detection server 402” (see, e.g., Aggarwal, paragraph 56). One of ordinary skill in the art would have been motivated to perform the method of Fu on a server using the teachings of Aggarwal because “conventional approaches for detecting outliers in a dataset involve manual or brute-force analysis of the dataset … [which] can be a daunting task …, particularly when operating under time and budget constraints” whereas Aggarwal’s “automated outlier detection system in accordance with example embodiments of the disclosure represents an improvement to existing outlier detection technology by providing a mechanism by which optimal exploratory actions can be identified, whose exploration has the greatest likelihood of yielding accurate outlier detection within specified time and budget constraints”, as suggested by Aggarwal. (see, e.g., Aggarwal, paragraphs 3 and 35).
With respect to independent claim 8, claim 8 is substantially similar to claim 1 and therefore is rejected on the same ground as claim 1, discussed above. In particular, claim 8 is an apparatus claim that performs operations that correspond to the method steps of claim 1.
Although Fu substantially discloses the claimed invention, Fu does not explicitly disclose a computing apparatus for selecting features within a data set to use for outlier detection analysis, the computing apparatus comprising: a processing entity operable to: <perform the above-noted steps/operations>.
However, in the same field, analogous art Aggarwal teaches a computing apparatus for selecting features within a data set to use for outlier detection analysis, the computing apparatus comprising: a processing entity10 operable to: <perform the above-noted steps/operations> (see, e.g., paragraph 68, “instructions may be loaded onto a special-purpose computer or other particular machine, a processor, or other programmable data processing apparatus to produce a particular machine, such that execution of the instructions on the computer, processor, or other programmable data processing apparatus causes one or more functions or operations specified in the flow diagrams to be performed. … The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process.”).
Fu and Aggarwal are analogous art because they are both directed to methods for outlier detection. (see, e.g., Fu, abstract; Aggarwal, abstract)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the disclosed method of Fu to incorporate the teachings of Aggarwal in order to have the method be “performed by an outlier detection server 402” (see, e.g., Aggarwal, paragraph 56). One of ordinary skill in the art would have been motivated to perform the method of Fu on a server using the teachings of Aggarwal because “conventional approaches for detecting outliers in a dataset involve manual or brute-force analysis of the dataset … [which] can be a daunting task …, particularly when operating under time and budget constraints” whereas Aggarwal’s “automated outlier detection system in accordance with example embodiments of the disclosure represents an improvement to existing outlier detection technology by providing a mechanism by which optimal exploratory actions can be identified, whose exploration has the greatest likelihood of yielding accurate outlier detection within specified time and budget constraints”, as suggested by Aggarwal. (see, e.g., Aggarwal, paragraphs 3 and 35).
With respect to independent claim 15, claim 15 is substantially similar to claim 1 and therefore is rejected on the same ground as claim 1, discussed above. In particular, claim 15 is a computer-readable media claim that performs operations that correspond to the method steps of claim 1.
Although Fu substantially discloses the claimed invention, Fu does not explicitly disclose Non-transitory computer-readable media containing a program element executable by a computing system to perform a method.
However, in the same field, analogous art Aggarwal teaches Non-transitory computer-readable media containing a program element executable by a computing system to perform a method (see, e.g., paragraphs 68, “computer program instructions may also be stored in a computer-readable storage medium (CRSM) that upon execution may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement one or more functions or operations specified in the flow diagram. … The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process.”).
Fu and Aggarwal are analogous art because they are both directed to methods for outlier detection. (see, e.g., Fu, abstract; Aggarwal, abstract)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the disclosed method of Fu to incorporate the teachings of Aggarwal in order to have the method be “performed by an outlier detection server 402” (see, e.g., Aggarwal, paragraph 56). One of ordinary skill in the art would have been motivated to perform the method of Fu on a server using the teachings of Aggarwal because “conventional approaches for detecting outliers in a dataset involve manual or brute-force analysis of the dataset … [which] can be a daunting task …, particularly when operating under time and budget constraints” whereas Aggarwal’s “automated outlier detection system in accordance with example embodiments of the disclosure represents an improvement to existing outlier detection technology by providing a mechanism by which optimal exploratory actions can be identified, whose exploration has the greatest likelihood of yielding accurate outlier detection within specified time and budget constraints”, as suggested by Aggarwal. (see, e.g., Aggarwal, paragraphs 3 and 35).
Regarding claims 2, 9 and 16, as discussed above, Fu in view of Aggarwal teaches the method of claim 1, the apparatus of claim 8, and the computer readable medium of claim 15.
Fu further discloses selecting the features within the data set to use for outlier detection analysis further comprises:
prior to selecting the one or more of the candidate features, determining one or more pairwise correlations between the candidate features (see, e.g., page 6, section 4.1, “Algorithm 1 begins by initializing Fnr to all features in A (line 1). It then iteratively looks for a member in Fnr which can be considered redundant and hence dropped [i.e., inhibiting using] from Fnr. In order for a feature A∗,p ∈ Fnr to be considered redundant, it needs to (i) be in a maximally correlated feature pair {A∗,p,A∗,q} ⊆ Fnr (line 4) [i.e., pairwise correlations], (ii) the correlation coefficient σp,q must be above the REMIX’s redundancy threshold α (line 5) [i.e., predetermined threshold], and (iii) of the two features in the pair, A∗,p must have a mean correlation with other features in Fnr that ≥ the mean correlation of A∗,q (line 7)”; and,
if one of the correlations, between two of the candidate features, is above a predetermined threshold, inhibiting using both of the candidate features associated with the one of the correlations (see, e.g., page 6, section 4.1, “Algorithm 1 begins by initializing Fnr to all features in A (line 1). It then iteratively looks for a member in Fnr which can be considered redundant and hence dropped [i.e., inhibiting using] from Fnr. In order for a feature A∗,p ∈ Fnr to be considered redundant, it needs to (i) be in a maximally correlated feature pair {A∗,p,A∗,q} ⊆ Fnr (line 4) [i.e., pairwise correlations], (ii) the correlation coefficient σp,q must be above the REMIX’s redundancy threshold α (line 5) [i.e., predetermined threshold], and (iii) of the two features in the pair, A∗,p must have a mean correlation with other features in Fnr that ≥ the mean correlation of A∗,q (line 7)”).
Regarding claims 4, 11 and 18, as discussed above, Fu in view of Aggarwal teaches the method of claim 1, the apparatus of claim 8, and the computer readable medium of claim 15.
Fu further discloses determining the information content metric11 for values of data entries within the criteria12 of the candidate feature comprises using entropy calculations on the values of data entries within the criteria (see, e.g., page 6, section 4.1, “The family of prioritized feature subspaces Fp is created as follows. The features in Fnr are first sorted according to their Laplacian scores … We add |Fnr| subspaces to F¬p, where the lth subspace is the set of top-features in Fnr ranked by their Laplacian scores”; page 5, algorithm 1, lines 14-18, the for loop is depicting this process of selecting features based on their calculated Laplacian scores; page 7, section 4.3, “These statistics include the Laplacian score …, and entropy which is a measure of the information content in the feature.” [i.e., Laplacian scores are used in algorithm 1 for the ranking, but can be substituted for entropy, corresponding to an information content metric]).
Regarding claims 5, 12 and 19, as discussed above, Fu in view of Aggarwal teaches the method of claim 1, the apparatus of claim 8, and the computer readable medium of claim 15.
Fu further discloses selecting a plurality of outlier detection operations to generate an outlier characteristic metric for evaluating data entries within the data set, each of the outlier detection operations comprising an outlier detection algorithm being run with one of the selected features (see, e.g., page 4, section 3.2, “An outlier detector Da,f is simply a combination of an algorithm a ∈ A and a feature subspace f of A … REMIX enumerates candidate outlier detectors based on a family of prioritized feature subspaces Fp and a family of randomly constructed feature subspaces Fr”; page 9, section 4.5, ““Each detector executed by REMIX provides an outlier score” [i.e., selecting candidate algorithm/outlier detector operations to be used for generating an outlier score/characteristic metric]);
generating an outlier characteristic metric for each one of a set of data items using each of the plurality of selected outlier detection operations, wherein each one of the set of data items is one of the plurality of data entries or a set of associated ones of the plurality of data entries; (see, e.g., page 9, section 4.5, “Each detector executed by REMIX provides an outlier score [i.e., outlier characteristic metric] for each data point” [i.e., data entries]); and
combining the plurality of outlier characteristic metrics for said each one of the set of data items to generate an ensemble outlier metric for said each one of the set of data items. (see, e.g., page 6, FIG.2 description, “The outlier results [i.e., outlier characteristic metrics] from the detectors executed by REMIX are factorized into perspectives, and also ensembled into a single set of results.”)
Claims 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Fu in view of Aggarwal as applied to claims 1, 8 and 15 above, and further in view of non-patent literature Ukil, Arijit ("Application of Kolmogorov complexity in anomaly detection." 2010 16th Asia-Pacific Conference on Communications (APCC). IEEE, 2010, hereinafter “Ukil”).
Regarding claims 3, 10 and 17, as discussed above, Fu in view of Aggarwal teaches the method according to claim 1, the apparatus of claim 8, and the computer readable media of claim 15.
However, Fu in view of Aggarwal does not explicitly teach wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises determining a potential compression rate for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature.
In the same field, analogous art Ukil teaches wherein the determining the information content metric for values of data entries within the criteria of the candidate feature comprises determining a potential compression rate13 for the values of data entries within the criteria, wherein the information content metric is a decreasing function of the potential compression rate for the values of data entries within the criteria of the candidate feature (see, e.g., page 141, Sect. I, “the Kolmogorov complexity approach we consider the individual objects themselves in isolation and the encoding of an object is a short computer program, which is considered as a compressed version of the object that generates it.” and page 144, right, “we formulate the algorithm. Firstly, we train the system with a large set of regular traffic. For every training traffic s, inverse Kolmogorov complexity (equivalent to inverse compression ratio) Ks is calculated [i.e., calculate/determine compression ratio for values of data entries in data traffic] … The sum of the inverse compression ratio of the individual packets is compared to the fully concatenated sequence. A complexity differential is defined” [i.e., potential compression ratio for data entries/packets is an inverse of/decreasing function Kolmogorov complexity]).
Fu, Aggarwal and Ukil are analogous art because they are each directed to methods for outlier/anomaly detection. (see, e.g., Fu, abstract; Aggarwal, abstract; and Ukil, abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the disclosed method of Fu in view of Aggarwal to incorporate the teachings of Ukil in order to provide a method “for anomaly detection” in “a signature based anomaly detection scheme” through “application of Kolmogorov complexity” where “Kolmogorov complexity is used in many applications like spam filtering, data compression” (see, e.g., Ukil, Abstract). One of ordinary skill in the art would have been motivated to perform the method of Fu in view of Aggarwal using the anomaly detection method and scheme of Ukil because “with the help of Kolmogorov complexity, we can detect traffic pattern abnormality in a simplistic way. This detection and quantification of traffic pattern eventually lead to anomaly detection” and in order “To form a simplified IDS or anomaly detector which with some probability can declare the nature of the traffic or the anomaly content of the entered traffic” which advantageously “decreases the load of the IDS [intrusion/anomaly detection system] of the system.” where “Kolmogorov Complexity used in this work is based upon a complexity differential. Events are represented as sequences of strings containing certain information. Sequences are basically IP packets. The sum of the inverse compression ratio of the individual packets is compared to the fully concatenated sequence. A complexity differential is defined as the difference”, as suggested by Ukil. (see, e.g., Ukil, Abstract and page 144, right side, Sect. III).
Claims 6, 7, 13, 14, 20, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over non-patent literature Fu in view of Aggarwal as applied to claims 1, 5, 8, 12, 15 and 19 above, and further in view of non-patent literature Shi et al. (“Selective Combination based on Diversity-Accuracy Balance in Outlier Ensembles”, 2020, hereinafter “Shi”).
Regarding claims 6, 13 and 20, as discussed above, Fu in view of Aggarwal teaches the method according to claim 5, the apparatus of claim 12, and the computer readable media of claim 19.
However, Fu in view of Aggarwal does not explicitly teach the selecting the plurality of outlier detection operations comprises: determining which of a plurality of candidate outlier detection algorithms can use the selected features;
running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results;
determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection;
and selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results.
In the same field, analogous art Shi teaches the selecting the plurality of outlier detection operations comprises: determining which of a plurality of candidate outlier detection algorithms can use the selected features (see, e.g., page 1274, right side, Sect. I, “a pool of base detectors and clusters outlier score matrix trained on the training dataset, and then the most competent detector is selected from each cluster … the proposed method is intuitive and effective for unsupervised combination of independent outlier detectors.” and 1275, Sect. II, “For the diversity of ensemble members, … it is possible to induce diversity by different parameters, training set or feature space … Feature Bagging(FB): The diversified set of base detectors are generated by training on randomly selected feature subsets.” [i.e., determine which candidate outlier detectors/detector algorithms can use selected features]);
running each of the candidate outlier detection algorithms with each of the selected features that can be used with the particular candidate outlier detection algorithm to generate candidate algorithm results (see, e.g., page 1276, left side, section A, “Training data Xtrain ∈ Rt× d denotes the data with t points and d features, the algorithm first generates a pool of base detectors [C = C1,C2,C3,,,Cr] with various hyperparemeters. The results of all base detectors trained on Xtrain are combined into an outlier score matrix O(Xtrain), shown in Eq.(1), let Fr denotes the score vectors from the rth base detector” [i.e., algorithms/detectors are executed/run to generate the results]);
determining an effectiveness metric for each of the candidate algorithm results, the effectiveness metric measuring the ability of the candidate algorithm results to separate data entries for outlier detection (see, e.g., page 1276, right side, algorithm 1, lines 4, “Evaluate the competency [i.e., effectiveness] of Cr [i.e., a candidate detector/algorithm] by the Pearson correlation between center point and Fr in each class”); and
selecting the plurality of candidate outlier detection algorithms with specific features to use as the selected outlier detection operations based at least partially on the effectiveness metrics for the corresponding candidate algorithm results (see, e.g., page 1276, right side, algorithm 1, line 5, “Selecting the most competent detector from every class with the highest Pearson correlation to center point to compose Cs =[C1,...,Ck]” [i.e., selecting candidate outlier detectors/algorithms based on effectiveness/competency metrics/measurements]).
Fu, Aggarwal and Shi are analogous art because they are each directed to methods for outlier detection. (see, e.g., Fu, abstract; Aggarwal, abstract; and Shi, abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the disclosed method of Fu in view of Aggarwal to incorporate the teachings of Shi in order to provide “a new approach for outlier detection by taking the balance between diversity and accuracy in unsupervised ensembles into account.” where a “clustering algorithm is adopted to induce the diversity of ensemble, and the most competent detector in each cluster is picked to improve the accuracy of ensemble” by using “a new unsupervised outlier detection method called Selective Combination based on Diversity-Accuracy Balance in Outlier Ensembles. The idea is motivated by the diversity metrics in supervised ensemble learning [8], where the diversity of ensemble is measured by a paired diversity and unpaired diversity.” (see, e.g., Shi, Abstract and page 1274, right side, Sect. I). One of ordinary skill in the art would have been motivated to perform the method of Fu in view of Aggarwal using the outlier detection method of Shi because Shi’s “proposed method outperforms these compared methods on the majority of datasets.”, “Compared to existing methods, the proposed method has less computing costs, and can be formulated to be compatible with different types of base detectors.” and “the proposed method outperforms other popular methods and is more effective for outlier detection”, as suggested by Shi. (see, e.g., Shi, Abstract and page 1274, right side, Sect. I).
Regarding claims 7, 14 and 21, as discussed above, Fu in view of Aggarwal and Shi teaches the method according to claim 6, the apparatus of claim 13, and the computer readable media of claim 20.
Fu further discloses “The interactive outlier exploration problem is a budgeted optimization problem which selects a subset of detectors with a maximization objective that is a linear combination of two quantities: (i) the total utility of all the selected detectors, and (ii) the total utility of the top-k selected detectors, subject to the following budget and diversity constraints”, “selects a subset of candidates for execution based on a utility maximization objective and budget and diversity constraints” and “the selection of base outlier detectors in a manner that is sensitive to their execution costs, while maximizing an aggregate utility function of the solution and also ensuring diversity across algorithmic choice” [i.e., determining diversity constraints/metrics] (see, e.g., pages 4, 6 and 13).
However, Fu in view of Aggarwal is not relied upon to explicitly teach wherein the selecting a plurality of outlier detection operations further comprises: determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results; and
wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results.
In the same field, analogous art Shi teaches wherein the selecting a plurality of outlier detection operations further comprises: determining a diversity metric for each of the candidate algorithm results relative to all other candidate algorithm results, the diversity metric measuring the correlation between information in the candidate algorithm results (see, e.g., page 1274, right, Sect. I, “we propose a new unsupervised outlier detection method called Selective Combination based on Diversity-Accuracy Balance in Outlier Ensembles … motivated by the diversity metrics in supervised ensemble learning [8], where the diversity of ensemble is measured by a paired diversity and unpaired diversity.”, pages 1275-1276, Sects II-III A, “For the diversity of ensemble members, … it is possible to induce diversity by different parameters, training set or feature space; about the combination of members to an ensemble, … by averaging or maximizing the all base detectors scores”, “induce diversity, such as subsampling the training set and feature space, or by setting different hyperparameters” [i.e., determine diversity metrics/parameters for the candidate algorithm results/scores relative to other candidate algorithm results in the ensemble, the metric measuring correlation between information in the candidate algorithm scores/results]); and
wherein the selecting the plurality of candidate outlier detection algorithms with specific features as the selected outlier detection operations is further based at least partially on the diversity metrics for the candidate algorithm results relative to other candidate algorithm results (see, e.g., page 1276, left, Sect. III A-B, “In an effective ensemble, diversified base detectors are important … induce diversity, such as subsampling the training set and feature space, or by setting different hyperparameters … Selective Combination based on Diversity-Accuracy Balance Consider alternative ensembles based on diversity, the O(Xtrain) can be grouped into k classes by clustering algorithm, and then in each cluster, the Pearson Correlation Coefficient [10] between the score of each detector and the center of the cluster was calculated” [i.e., selecting candidate outlier detectors/algorithms with specific features in the feature space] and 1277, right Sect. IV B, “select detectors based on the balance between diversity and accuracy.” [i.e., select the candidate outlier detectors/algorithms based at least partially on the diversity metrics/parameters]).
The motivation to combine Fu, Aggarwal and Shi is the same as discussed above with respect to claims 6, 13 and 20.
Conclusion
The prior art made of record, listed on form PTO-892, and not relied upon, is considered pertinent to applicant's disclosure.
The references listed on form PTO-892 are all generally related to techniques, methods and systems for using artificial intelligence/AI and machine learning/ML models to detect outliers and anomalies.
For example, Akoglu et al. (U.S. Patent Application Pub. No. 2014/0074838 A1, hereinafter “Akoglu”) teaches scoring database attribute values for anomalousness using a compression-cost metric derived from dictionary-based data compression across attribute groupings. (see, e.g., Abstract and paragraphs 15, 29, 34, 54-56 and 59-60).
Also, for example, Givental et al. (U.S. Patent Application Pub. No. 20210279644 A1, hereinafter “Givental”) teaches an ensemble of unsupervised models generating a weighted-combination anomaly score, with weights dynamically adjusted from analyst correctness feedback. (see, e.g., Abstract and paragraphs 26, 53 and 57).
Further, for instance, non-patent literature Das et al., ("Incorporating Expert Feedback into Active Anomaly Discovery," IEEE ICDM 2016, pp. 853–858, hereinafter “Das”) teaches an ensemble of anomaly detectors combined via a weighted linear score function, with weights learned through optimization responsive to analyst-labeled anomaly/nominal feedback. (see, e.g., Abstract and pages 854-855, 856 and 858).
Additionally, for instance, non-patent literature Li et al. ("An Ensemble Outlier Detection Method Based on Information Entropy-Weighted Subspaces for High-Dimensional Data," Entropy 2023, 25(8), 1185, hereinafter “Li”) teaches computing information entropy per dimension to weight feature subspaces combined with ensemble outlier detectors on data subsamples. (see, e.g., Abstract and pages 4-9).
The examiner requests, in response to this office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the reference cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111 (c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RANDY K BALDWIN whose telephone number is (571)270-5222. The examiner can normally be reached on Mon - Fri 9:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RANDALL K. BALDWIN/Primary Examiner, Art Unit 2125
1 As noted in the section 112(b) rejection of this claim above, the “processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
2 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
3 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
4 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
5 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
6 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
7 As noted in the section 112(b) rejection of this claim above, “the processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
8 Paragraph 0051 of the specification states that “a criterion may be a field or portion of the data entry which is associated with a particular type of information … a criterion can be a characteristic of a data set as a whole.” Thus, criteria, under the BRI, in view of the specification are any data fields or an overall characteristic of a data set as a whole.
9 Paragraph 75 of the specification states that “information content metric can measure the amount of information contained” and paragraph 76 states that “a variety of information content metrics can be used”. Thus, information content metric, under the BRI, in view of the specification is any metric, measurement, or score associated with or corresponding to information content.
10 As noted in the section 112(b) rejection of this claim above, the “processing entity” has been interpreted as any combination of software (i.e., a set of instructions, code, one or more functions or software modules) and/or hardware (i.e., circuitry and/or hardware logic components/modules, such as, but not limited to the “processor 1110” depicted in FIG. 11) capable of performing the claimed functions.
11 As noted above, information content metric is any metric, measurement, or score associated with or corresponding to information content
12 As noted above, criteria under the BRI are any data fields or an overall characteristic of a data set as a whole
13 Paragraph 78 of the specification states “compute potentially (e.g. using the associated probability distribution defined above) or test empirically how much this data can be compressed. The information content metric may be measured as the rate of compression that can be achieved on a given feature or criterion. … information content metrics can be generated based on the result of an algorithm as well as the input features … potential compression rate can be defined as the ratio between an uncompressed data size and a compressed data size”. Thus “potential compression rate”, under the BRI, in view of the specification, is any possible or potential rate, ratio or amount of data compression.