DETAILED ACTION
This Action is a response to the filing received 20 June 2024. Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 20 June 2024 is being considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 7 and 11 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Goyal et al., U.S. 2018/0337840 A1 (“Goyal”).
Regarding claim 1, Goyal teaches: A method comprising: receiving, at a computer system of an event processing system during a live test of changes to a current set of event processing filters, event details for a live event being processed by the event processing system (Goyal, e.g., ¶¶24-25, “PubSub communication pattern may allow for real-time communication between publishers and subscribers … referred to herein as a real-time messaging (RTM) system … filtering messages in a PubSub messaging system may include processing messages … in real-time … Testing filtering may also be challenging … providing time-efficient, processing-efficient, and resource-efficient systems and methods for testing filtering of data streams …”);
determining, by the computer system, a selected set of event processing filters with which to evaluate the live event, wherein the determining is performed between a plurality of sets of event processing filters that includes the current set of event processing filters and a first new set of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system));
evaluating, by the computer system, the live event according to the selected set of event processing filters (Goyal, e.g., ¶137, “Testing system 800 may also include RTM system … RTM system 806 may execute test cases, the results of which (actual data sets) may later be compared to the expected results (e.g., the oracle) generated from test cases executed by test engine …”); and
updating, by the computer system based on the evaluating, metrics for events evaluated during the live test, the metrics including: a first set of metrics for the events evaluated during the live test using the current set of event processing filters; and a second set of metrics for the events evaluated during the live test using the first new set of event processing filters (Goyal, e.g., ¶138, “after test engine 804 performs a test (e.g., from database 802), and RTM system 806 performs a corresponding test, test engine 804 may generate a test report 810 including one or more performance characteristics of the test case (data stream filter). The performance characteristics may represent an analysis of the comparison of the expected results to the actual result … may indicate how closely the actual results are to the expected results …”);
wherein the first set of metrics and the second set of metrics are updated using information generated external to the event processing system (Goyal, e.g., FIG. 1A, displaying that the event processing system (100) receives and acts on information generated external to the event processing system, such as by publishers 1-N).
Regarding claim 7, the rejection of claim 1 is incorporated, and Goyal further teaches: wherein the plurality of sets of event processing filters includes a second new set of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system)), the metrics including a third set of metrics for events evaluated during the live test using the second new set of events processing filters (Goyal, e.g., ¶138, “after test engine 804 performs a test (e.g., from database 802), and RTM system 806 performs a corresponding test, test engine 804 may generate a test report 810 including one or more performance characteristics of the test case (data stream filter). The performance characteristics may represent an analysis of the comparison of the expected results to the actual result … may indicate how closely the actual results are to the expected results …”).
Regarding claim 11, Goyal teaches: A non-transitory, computer-readable medium storing program instructions that are executable on a computer system of an event processing system to perform operations (Goyal, e.g., ¶161, “The term ‘computer-readable storage medium’ shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform the methods described herein …”) that comprise: [[[the steps of the method of claim 1]]].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Mainguet et al., U.S. 2008/0226135 A1 (“Mainguet”).
Regarding claim 2, the rejection of claim 1 is incorporated, and Goyal further teaches: wherein the selected set of event processing filters includes a particular event processing filter that is not present in any of remaining ones of the plurality of sets of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system, and may be distinct from any other filters)).
Goyal does not more particularly teach that evaluating the live event includes determining whether to allow a transaction based on the live event to be processed based on results of applying the selected set of event processing filters to the live event. However, Mainguet does teach: wherein evaluating the live event includes determining whether to allow a transaction based on the live event to be processed based on results of applying the selected set of event processing filters to the live event (Mainguet, e.g., ¶67, “reconstruction and filtering engine 504 can … store biometric data … filtering buffer 508 can, for example, be configured to store representative data … based on the biometric data …” See also, e.g., ¶¶84-85, “If a recognition has occurred, then stage 714 executes a secured process … allow a secured transaction … reconstruction and filtering engine 504 and/or the recognition engine 510 can determine whether a sensing event has ended …”) for the purpose of receiving, storing and evaluating incoming event data in order to determine whether a secured transaction should be permitted to proceed (Mainguet, e.g., ¶¶66-87).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that evaluating the live event includes determining whether to allow a transaction based on the live event to be processed based on results of applying the selected set of event processing filters to the live event because the disclosure of Mainguet shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for event processing and evaluation to provide that evaluating the live event includes determining whether to allow a transaction based on the live event to be processed based on results of applying the selected set of event processing filters to the live event for the purpose of receiving, storing and evaluating incoming event data in order to determine whether a secured transaction should be permitted to proceed (Mainguet, Id.).
Claims 3 and 15 are rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi et al., U.S. 10,262,271 B1 (“Doddi”).
Regarding claim 3, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach that over a duration of the live test, the system splits a stream of live events into at least two sub-streams including a first sub-stream evaluated by the current event processing filters and a second sub-stream evaluated by the second filters, wherein the stream is split according to a split parameter. However, Doddi does teach: wherein, over a duration of the live test, the computer system splits a stream of live events that includes the live event into at least two sub-streams that includes a first sub-stream evaluated according to the current set of event processing filters and a second sub-stream evaluated according to the first new set of event processing filters (Doddi, e.g., 7:56-65, “users can select the challenger button … instantiates a challenger module … provides model testing and comparison functionality such that data streams can be split across different models, run in parallel … challenger module provides for split testing, bucket testing, or other forms of AB testing …”), wherein the stream is split according to a specified split parameter (Doddi, e.g., 7:65-8:3, “users can request and specify which model instances to target with specific data sets and data sources …”) for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, e.g., 6:62-8:32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that over a duration of the live test, the system splits a stream of live events into at least two sub-streams including a first sub-stream evaluated by the current event processing filters and a second sub-stream evaluated by the second filters, wherein the stream is split according to a split parameter because the disclosure of Doddi shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for developing and testing machine-learning based classifiers to provide that over a duration of the live test, the system splits a stream of live events into at least two sub-streams including a first sub-stream evaluated by the current event processing filters and a second sub-stream evaluated by the second filters, wherein the stream is split according to a split parameter for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, Id.).
Regarding claim 15, the rejection of claim 11 is incorporated, and Goyal further teaches: wherein the plurality of sets of event processing filters further includes a second new set of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system, and may be distinct from any other filters)).
Goyal does not more particularly teach that the determining includes splitting, according to split parameters, live events in the test between the current set of event processing filters, a first new set of filters, and a second new set of filters. However, Doddi does teach: wherein the determining includes splitting, according to split parameters (Doddi, e.g., 7:65-8:3, “users can request and specify which model instances to target with specific data sets and data sources …”), live events in the live test between the current set of event processing filters, a first new set of event processing filters, and a second new set of event processing filters (Doddi, e.g., 7:56-65, “users can select the challenger button … instantiates a challenger module … provides model testing and comparison functionality such that data streams can be split across different models, run in parallel … challenger module provides for split testing, bucket testing, or other forms of AB testing …”) for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, e.g., 6:62-8:32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that the determining includes splitting, according to split parameters, live events in the test between the current set of event processing filters, a first new set of filters, and a second new set of filters because the disclosure of Doddi shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for developing and testing machine-learning based classifiers to provide that the determining includes splitting, according to split parameters, live events in the test between the current set of event processing filters, a first new set of filters, and a second new set of filters for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, Id.).
Claim 4 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi, and in further view of Nychis et al., U.S. 11,816,112 B1 (“Nychis”) and Takahashi et al., U.S. 2018/0124774 A1 (“Takahashi”).
Regarding claim 4, the rejection of claim 3 is incorporated, but Goyal in view of Doddi does not more particularly teach that the selected set of filters is determined for the live event by hashing metadata in the event details into a particular bin corresponding to the first new set of processing filters. However, Nychis does teach: wherein the selected set of processing filters is determined for the live event by hashing metadata in the event details into a particular one of a plurality of bins corresponding to the first new set of processing filters (Nychis, e.g., 23:4-14, “one or more features may be determined for events in a collected stream … features may be one or more hashes of metadata associated with the events in the collected stream … After capturing the stream of events in act 430 … one or more features (e.g., hashes) are determined for metadata associated with events in the stream of events … a signature for a process is generated based on the features …” See also, e.g., 23:29-37, “at least some events that are captured in a stream of events are tagged with a process name and an instance identifier, which enables the events to be sorted for further processing … events tagged with a particular process name may be filtered by instance identifier …”) for the purpose of analyzing live incoming stream data in order to identify and then tag process instances, such that processes can be later identified and filtered for further analysis (Nychis, e.g., 22:53-26:62).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal in view of Doddi to provide that the selected set of filters is determined for the live event by hashing metadata in the event details into a particular bin corresponding to the first new set of processing filters because the disclosure of Nychis shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for stream data evaluation to automatically identify process data to provide that the selected set of filters is determined for the live event by hashing metadata in the event details into a particular bin corresponding to the first new set of processing filters for the purpose of analyzing live incoming stream data in order to identify and then tag process instances, such that processes can be later identified and filtered for further analysis (Nychis, Id.).
Goyal in view of Doddi and Nychis does not more particularly teach that a size of the particular bin is determined based on the specified split parameter. However, Takahashi does teach: a size of the particular bin being determined based on the specified split parameter (Takahashi, e.g., ¶¶206-208, “transmit category information of the user apparatus to the base station … receive a parameter including a transmission mode … soft buffer size determination means configured, if the category information is a value equal to or greater than a predetermined value and if the parameter is a predetermined parameter … to determine a soft buffer size corresponding to the category information …”) for the purpose of ensuring that the user device can easily determine a buffer size (Takahashi, e.g., ¶¶205-218).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal in view of Doddi and Nychis to provide that a size of the particular bin is determined based on the specified split parameter because the disclosure of Takahashi shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for mobile device communication and incoming event processing and storage to provide that a size of the particular bin is determined based on the specified split parameter for the purpose of ensuring that the user device can easily determine a buffer size (Takahashi, Id.).
Claim 5 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Cable et al., U.S. 2025/0123947 A1 (“Cable”).
Regarding claim 5, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach receiving from a user computing device, a request to render on the computing device a dashboard displaying visual representations of the first and second sets of metrics. However, Cable does teach: receiving, from a computing device of a user of the event processing system, a request to render, on the computing device, a dashboard that displays visual representations of the first set of metrics and the second set of metrics (Cable, e.g., ¶5, “in response to receiving a report request from the user device … transmitting, for display on the user device, the first report.” See also, e.g., ¶112, “output generation module 628 is configured to generate various outputs to present the results to the requesting device … output generation module 628 may generate … a dashboard summary … a Looker dashboard for trend analysis”) for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, e.g., ¶¶111-114).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide for receiving from a user computing device, a request to render on the computing device a dashboard displaying visual representations of the first and second sets of metrics because the disclosure of Cable shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for data receipt, evaluation, categorization and analysis to provide for receiving from a user computing device, a request to render on the computing device a dashboard displaying visual representations of the first and second sets of metrics for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, Id.).
Claims 6, 8, 12 and 14 are rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Sneed et al., U.S. 2021/0406150 A1 (“Sneed”).
Regarding claim 6, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach receiving from a user computing device an indication to promote the first new set of processing filters to become the current set of filters. However, Sneed does teach: receiving, from a computing device of a user of the event processing system, an indication to promote the first new set of event processing filters to become the current set of event processing filters (Sneed, e.g., ¶43, “Once sampled signals are available under the debugger module, a developer may use a browser application and develop (320) one or more filter functions … After one or more iterations of debugging the custom code, the developer may save (325) the debugged one or more filter functions in the server-side developer console for various performance and security benchmarking tests … Post security and performance checks, the tested one or more filter functions are stored (330) in the code cache for client downloading by the integrated client application via the fetch routine …”) for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, e.g., ¶¶43-54).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide for receiving from a user computing device an indication to promote the first new set of processing filters to become the current set of filters because the disclosure of Sneed shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for configurable application instrumentation and event processing and filtering to provide for receiving from a user computing device an indication to promote the first new set of processing filters to become the current set of filters for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, Id.).
Regarding claim 8, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach making further updates to the metrics based on information regarding the live event that is received after the live test concludes. However, Sneed does teach: making further updates to the metrics based on information regarding the live event that is received after conclusion of the live test (Sneed, e.g., ¶43, “tested one or more filter functions are stored (330) in the client cache for downloading by the integrated client application via the fetch runtime. The downloaded one or more filter functions are loaded (335) into the function runtime …” See also, e.g., ¶44, “loads (405) a copy of one or more filter functions to memory and may start passing (410) each signal in the buffer through one or more filter functions …”) for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, e.g., ¶¶43-54).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide for making further updates to the metrics based on information regarding the live event that is received after the live test concludes because the disclosure of Sneed shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for configurable application instrumentation and event processing and filtering to provide for making further updates to the metrics based on information regarding the live event that is received after the live test concludes for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, Id.).
Regarding claim 12, the rejection of claim 11 is incorporated, but Goyal does not more particularly teach receiving from a user first input data causing the system to commence the live test, and receiving second input data causing the computer system to promote the first set of new event processing filters to the current set of filters for subsequent live event evaluation. However, Sneed does teach: receiving, from a user of the event processing system, first input data that causes the computer system to commence the live test; and receiving, from the user after conclusion of the live test, second input data which causes the computer system to promote the first set of new event processing filters to the current set of event processing filters for subsequent evaluation of live events (Sneed, e.g., ¶43, “Once sampled signals are available under the debugger module, a developer may use a browser application and develop (320) one or more filter functions … After one or more iterations of debugging the custom code, the developer may save (325) the debugged one or more filter functions in the server-side developer console for various performance and security benchmarking tests … Post security and performance checks, the tested one or more filter functions are stored (330) in the code cache for client downloading by the integrated client application via the fetch routine …”) for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, e.g., ¶¶43-54).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide for receiving from a user first input data causing the system to commence the live test, and receiving second input data causing the computer system to promote the first set of new event processing filters to the current set of filters for subsequent live event evaluation because the disclosure of Sneed shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for configurable application instrumentation and event processing and filtering to provide for receiving from a user first input data causing the system to commence the live test, and receiving second input data causing the computer system to promote the first set of new event processing filters to the current set of filters for subsequent live event evaluation for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, Id.).
Regarding claim 14, the rejection of claim 11 is incorporated, but Goyal does not more particularly teach that the operations comprise continuing to update the first and second sets of metrics in response to receiving additional live event information after the live test concludes. However, Sneed does teach: wherein the operations further comprise continuing to update the first set of metrics and set of metrics in response to receiving, after conclusion of the live test, additional information about live events in the live test (Sneed, e.g., ¶43, “tested one or more filter functions are stored (330) in the client cache for downloading by the integrated client application via the fetch runtime. The downloaded one or more filter functions are loaded (335) into the function runtime …” See also, e.g., ¶44, “loads (405) a copy of one or more filter functions to memory and may start passing (410) each signal in the buffer through one or more filter functions …”) for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, e.g., ¶¶43-54).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that the operations comprise continuing to update the first and second sets of metrics in response to receiving additional live event information after the live test concludes because the disclosure of Sneed shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for configurable application instrumentation and event processing and filtering to provide that the operations comprise continuing to update the first and second sets of metrics in response to receiving additional live event information after the live test concludes for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, Id.).
Claim 9 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Hencinski et al., U.S. 2022/0269780 A1 (“Hencinski”).
Regarding claim 9, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach that the event processing system is configured to classify computer security alerts within a computer network, the live event corresponding to a particular security event. However, Hencinski does teach: wherein the event processing system is a system configured to classify computer security alerts within a computer network, the live event corresponding to a particular security event (Hencinski, e.g., ¶38, “security alert engine 110 may include a security alert machine learning system 114 that may function to classify inbound security alerts …” See also, e.g., ¶52, “data signals generated from the one or more distinct security data sources may be processed through an automated security alert engine … that may or may not produce system-validated security alerts …”) for the purpose of classifying incoming security event alerts using a variety of machine-learning based classification methods in order to validate the type of alert and any action needed in response thereto (Hencinski, e.g., ¶¶38-60).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that the event processing system is configured to classify computer security alerts within a computer network, the live event corresponding to a particular security event because the disclosure of Hencinski shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for security event detection, filtering and analysis to provide that the event processing system is configured to classify computer security alerts within a computer network, the live event corresponding to a particular security event for the purpose of classifying incoming security event alerts using a variety of machine-learning based classification methods in order to validate the type of alert and any action needed in response thereto (Hencinski, Id.).
Claim 10 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Phan et al., U.S. 2022/0066913 A1 (“Phan”).
Regarding claim 10, the rejection of claim 1 is incorporated, but Goyal does not more particularly teach that the event processing system is configured to evaluate e-commerce transactions, the live event corresponding to a particular e-commerce transaction. However, Phan does teach: wherein the event processing system is a system configured to evaluate e-commerce transactions, the live event corresponding to a particular e-commerce transaction (Phan, e.g., ¶6, “providing a design specification for event data … including at least one of an event data source, an event data filter … filtering the retrieved testing events based in the design specification … generating analysis data by applying testing metrics to the configured event data …” See also, e.g., ¶¶22-24, “systems include a shipment authority technology (SAT) system 101, an external front end system 103, an internal front end system 105 … shipment and order tracking (SOT) system … in embodiments where system 100 enables the presentation of systems to enable users to place an order for an item …”) for the purpose of testing and evaluating event data in an e-commerce system based on varying attributes of system components and the data generated, collected and analyzed (Phan, e.g., ¶¶2-8, 68-78).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide that the event processing system is configured to evaluate e-commerce transactions, the live event corresponding to a particular e-commerce transaction because the disclosure of Phan shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for providing event data analysis functionality testing to provide that the event processing system is configured to evaluate e-commerce transactions, the live event corresponding to a particular e-commerce transaction for the purpose of testing and evaluating event data in an e-commerce system based on varying attributes of system components and the data generated, collected and analyzed (Phan, Id.).
Claim 13 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Sneed and Cable.
Regarding claim 13, the rejection of claim 12 is incorporated, but Goyal in view of Sneed does not more particularly teach that the operations further comprise receiving from the user a request to view a dashboard displaying a visual representation of the first and second sets of metrics. However, Cable does teach: wherein the operations further comprise: receiving, from the user, a request to view a dashboard that displays visual representations of the first set of metrics and the second set of metrics (Cable, e.g., ¶5, “in response to receiving a report request from the user device … transmitting, for display on the user device, the first report.” See also, e.g., ¶112, “output generation module 628 is configured to generate various outputs to present the results to the requesting device … output generation module 628 may generate … a dashboard summary … a Looker dashboard for trend analysis”) for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, e.g., ¶¶111-114).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal in view of Sneed to provide that the operations further comprise receiving from the user a request to view a dashboard displaying a visual representation of the first and second sets of metrics because the disclosure of Cable shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for data receipt, evaluation, categorization and analysis to provide that the operations further comprise receiving from the user a request to view a dashboard displaying a visual representation of the first and second sets of metrics for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, Id.).
Claim 16 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi and Cable.
Regarding claim 16, Goyal teaches: A method, comprising: performing, by a computer system of an event processing system (Goyal, e.g., ¶¶24-25, “PubSub communication pattern may allow for real-time communication between publishers and subscribers … referred to herein as a real-time messaging (RTM) system … filtering messages in a PubSub messaging system may include processing messages … in real-time … Testing filtering may also be challenging … providing time-efficient, processing-efficient, and resource-efficient systems and methods for testing filtering of data streams …”), a live test of a plurality of sets of event processing filters that includes a current set of event processing filters and a first new set of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system)), wherein the performing includes: …
evaluating the first set of live event data using the current set of event processing filters (Goyal, e.g., ¶137, “Testing system 800 may also include RTM system … RTM system 806 may execute test cases, the results of which (actual data sets) may later be compared to the expected results (e.g., the oracle) generated from test cases executed by test engine …”);
updating a first set of metrics for the first set of live event data (Goyal, e.g., ¶138, “after test engine 804 performs a test (e.g., from database 802), and RTM system 806 performs a corresponding test, test engine 804 may generate a test report 810 including one or more performance characteristics of the test case (data stream filter). The performance characteristics may represent an analysis of the comparison of the expected results to the actual result … may indicate how closely the actual results are to the expected results …”);
evaluating the second set of live event data using the first new set of event processing filters (Goyal, e.g., ¶137, “Testing system 800 may also include RTM system … RTM system 806 may execute test cases, the results of which (actual data sets) may later be compared to the expected results (e.g., the oracle) generated from test cases executed by test engine …”); and
updating a second set of metrics for the second set of live event data (Goyal, e.g., ¶138, “after test engine 804 performs a test (e.g., from database 802), and RTM system 806 performs a corresponding test, test engine 804 may generate a test report 810 including one or more performance characteristics of the test case (data stream filter). The performance characteristics may represent an analysis of the comparison of the expected results to the actual result … may indicate how closely the actual results are to the expected results …”); and …
wherein the first set of metrics and the second set of metrics are updated using information generated external to the event processing system (Goyal, e.g., FIG. 1A, displaying that the event processing system (100) receives and acts on information generated external to the event processing system, such as by publishers 1-N).
Goyal does not more particularly teach splitting, according to a split parameter, live event data received during the live test into first and second sets of live event data. However, Doddi does teach: splitting, according to a specified split parameter (Doddi, e.g., 7:65-8:3, “users can request and specify which model instances to target with specific data sets and data sources …”), live event data received during the live test into a first set of live event data and a second set of live event data (Doddi, e.g., 7:56-65, “users can select the challenger button … instantiates a challenger module … provides model testing and comparison functionality such that data streams can be split across different models, run in parallel … challenger module provides for split testing, bucket testing, or other forms of AB testing …”) for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, e.g., 6:62-8:32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal to provide for splitting, according to a split parameter, live event data received during the live test into first and second sets of live event data because the disclosure of Doddi shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for developing and testing machine-learning based classifiers to provide for splitting, according to a split parameter, live event data received during the live test into first and second sets of live event data for the purpose of permitting a user to run a customized test comparing a specified number of classifiers in their ability to classify data from one or more identified sources, run the tests in parallel, and accept or re-engineer one or more of the tested classifiers (Doddi, Id.).
Goyal in view of Doddi does not more particularly teach receiving a user request by the event processing system to render on a display a dashboard including visual representations of metrics including the first and second sets of metrics. However, Cable does teach: receiving, by the computer system, a request from a computing device of a user of the event processing system to render, on a display of the computing device, a dashboard that includes visual representations of a set of metrics that includes the first set of metrics and the second set of metrics (Cable, e.g., ¶5, “in response to receiving a report request from the user device … transmitting, for display on the user device, the first report.” See also, e.g., ¶112, “output generation module 628 is configured to generate various outputs to present the results to the requesting device … output generation module 628 may generate … a dashboard summary … a Looker dashboard for trend analysis”) for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, e.g., ¶¶111-114).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal in view of Doddi to provide for receiving a user request by the event processing system to render on a display a dashboard including visual representations of metrics including the first and second sets of metrics because the disclosure of Cable shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for data receipt, evaluation, categorization and analysis to provide for receiving a user request by the event processing system to render on a display a dashboard including visual representations of metrics including the first and second sets of metrics for the purpose of providing a variety of detailed output reports in response to a series of rule-based evaluations of received software code data (Cable, Id.).
Claim 17 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi and Cable, and in further view of Mainguet.
Regarding claim 17, the rejection of claim 16 is incorporated, but Goyal in view of Doddi and Cable does not more particularly teach that evaluating the first and second live event data sets includes determining whether to process transactions corresponding to live events in respective first and second sets of live data by the current set and the new set of event processing filters respectively. However, Mainguet does teach: wherein evaluating the first set of live event data includes determining whether to process transactions corresponding to live events in the first set of live event data using the current set of event processing filters, and wherein evaluating the second set of live event data includes determining whether to process transactions corresponding to live events in the second set of live event data using the first new set of event processing filters (Mainguet, e.g., ¶67, “reconstruction and filtering engine 504 can … store biometric data … filtering buffer 508 can, for example, be configured to store representative data … based on the biometric data …” See also, e.g., ¶¶84-85, “If a recognition has occurred, then stage 714 executes a secured process … allow a secured transaction … reconstruction and filtering engine 504 and/or the recognition engine 510 can determine whether a sensing event has ended …”) for the purpose of receiving, storing and evaluating incoming event data in order to determine whether a secured transaction should be permitted to proceed (Mainguet, e.g., ¶¶66-87).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal in view of Doddi and Cable to provide that evaluating the first and second live event data sets includes determining whether to process transactions corresponding to live events in respective first and second sets of live data by the current set and the new set of event processing filters respectively because the disclosure of Mainguet shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for event processing and evaluation to provide that evaluating the first and second live event data sets includes determining whether to process transactions corresponding to live events in respective first and second sets of live data by the current set and the new set of event processing filters respectively for the purpose of receiving, storing and evaluating incoming event data in order to determine whether a secured transaction should be permitted to proceed (Mainguet, Id.).
Claims 18-19 are rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi and Cable, and in further view of Mainguet and Nychis.
Regarding claim 18, the rejection of claim 17 is incorporated, and Doddi further teaches: wherein the splitting includes: receiving, during the live test, particular live event data for a particular live event (Doddi, e.g., 7:56-65, “users can select the challenger button … instantiates a challenger module … provides model testing and comparison functionality such that data streams can be split across different models, run in parallel … challenger module provides for split testing, bucket testing, or other forms of AB testing …”).
Goyal and Doddi in view of Cable and Mainguet do not more particularly teach hashing metadata in particular live event data into a particular bin corresponding to a particular set of event processing filters, the particular bin indicating the particular set of filters to evaluate the particular live event data, wherein the hashing is performed to achieve a percentage distribution of live event data indicated by the split parameter. However, Nychis does teach: hashing metadata in the particular live event data into a particular one of a plurality of bins corresponding to the plurality of sets of event processing filters, the particular bin indicating which set of event processing filters is to be used to evaluate the particular live event data, wherein the hashing is performed to achieve a percentage distribution of live event data that is indicated by the specified split parameter (Nychis, e.g., 23:4-14, “one or more features may be determined for events in a collected stream … features may be one or more hashes of metadata associated with the events in the collected stream … After capturing the stream of events in act 430 … one or more features (e.g., hashes) are determined for metadata associated with events in the stream of events … a signature for a process is generated based on the features …” See also, e.g., 23:29-37, “at least some events that are captured in a stream of events are tagged with a process name and an instance identifier, which enables the events to be sorted for further processing … events tagged with a particular process name may be filtered by instance identifier …”) for the purpose of analyzing live incoming stream data in order to identify and then tag process instances, such that processes can be later identified and filtered for further analysis (Nychis, e.g., 22:53-26:62).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal and Doddi in view of Cable and Mainguet to provide for hashing metadata in particular live event data into a particular bin corresponding to a particular set of event processing filters, the particular bin indicating the particular set of filters to evaluate the particular live event data, wherein the hashing is performed to achieve a percentage distribution of live event data indicated by the split parameter because the disclosure of Nychis shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for stream data evaluation to automatically identify process data to provide for hashing metadata in particular live event data into a particular bin corresponding to a particular set of event processing filters, the particular bin indicating the particular set of filters to evaluate the particular live event data, wherein the hashing is performed to achieve a percentage distribution of live event data indicated by the split parameter for the purpose of analyzing live incoming stream data in order to identify and then tag process instances, such that processes can be later identified and filtered for further analysis (Nychis, Id.).
Regarding claim 19, the rejection of claim 18 is incorporated, and Goyal and Doddi further teach: wherein the plurality of sets of event processing filters includes a second new set of event processing filters (Goyal, e.g., ¶134, “Database 802 may … store information, such as tests (e.g., test cases corresponding to data stream filters) and data sets to be tested …” See also, e.g., ¶135, “Test engine 804 may receive tests and/or data sets from database 802 and perform the tests …” Examiner’s note: by receiving the tests (corresponding to data stream filters), the selected set of filters is those that are received, and it is determined to use those instead of others; in some cases the filters are current filters (i.e., ones that already have test results and/or test data associated therewith) and in other cases they are new filters (i.e., ones that will be tested on live messages via the RTM system)), wherein the performing includes:
splitting, according to the specified split parameter, live event data received during the live test into the first set of live event data, the second set of live event data, and a third set of live event data (Doddi, e.g., 7:56-65, “users can select the challenger button … instantiates a challenger module … provides model testing and comparison functionality such that data streams can be split across different models, run in parallel … challenger module provides for split testing, bucket testing, or other forms of AB testing …”);
evaluating the third set of live event data using a second new set of event processing filters; and updating a third set of metrics for the third set of live event data (Goyal, e.g., ¶138, “after test engine 804 performs a test (e.g., from database 802), and RTM system 806 performs a corresponding test, test engine 804 may generate a test report 810 including one or more performance characteristics of the test case (data stream filter). The performance characteristics may represent an analysis of the comparison of the expected results to the actual result … may indicate how closely the actual results are to the expected results …”).
Claim 20 is rejected under 35 U.S.C. § 103 as being unpatentable over Goyal in view of Doddi and Cable, and in further view of Mainguet and Sneed.
Regarding claim 20, the rejection of claim 17 is incorporated, but Goyal and Doddi in view of Cable and Mainguet do not more particularly teach updating the first and second sets of metrics with additional live event information after the live test has ended. However, Sneed does teach: updating, after the live test has ended, the first set of metrics and the second set of metrics with additional information about live events in the live test (Sneed, e.g., ¶43, “tested one or more filter functions are stored (330) in the client cache for downloading by the integrated client application via the fetch runtime. The downloaded one or more filter functions are loaded (335) into the function runtime …” See also, e.g., ¶44, “loads (405) a copy of one or more filter functions to memory and may start passing (410) each signal in the buffer through one or more filter functions …”) for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, e.g., ¶¶43-54).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system and method for testing information filters as taught by Goyal and Doddi in view of Cable and Mainguet to provide for updating the first and second sets of metrics with additional live event information after the live test has ended because the disclosure of Sneed shows that it was known to those of ordinary skill in the pertinent art to improve a system and method for configurable application instrumentation and event processing and filtering to provide for updating the first and second sets of metrics with additional live event information after the live test has ended for the purpose of collecting and filtering data for detailed backend analysis, while providing for the ability to sample filtered event data in order for a developer to create or to debug new or existing filter functions prior to deploying them for additional event processing (Sneed, Id.).
Conclusion
Examiner has identified particular references contained in the prior art of record within the body of this action for the convenience of Applicant. Although the citations made are representative of the teachings in the art and are applied to the specific limitations within the enumerated claims, the teaching of the cited art as a whole is not limited to the cited passages. Other passages and figures may apply. Applicant, in preparing the response, should consider fully the entire reference as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art and/or disclosed by Examiner.
Examiner respectfully requests that, in response to this Office Action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist Examiner in prosecuting the application.
When responding to this Office Action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 C.F.R. 1.111(c).
Examiner interviews are available via telephone and video conferencing using a USPTO-supplied web-based collaboration tool. Applicant is encouraged to submit an Automated Interview Request (AIR) which may be done via https://www.uspto.gov/patent/uspto-automated-interview-request-air-form, or may contact Examiner directly via the methods below.
Any inquiry concerning this communication or earlier communication from Examiner should be directed to Andrew M. Lyons, whose telephone number is (571) 270-3529, and whose fax number is (571) 270-4529. The examiner can normally be reached Monday to Friday from 10:00 AM to 6:00 PM ET. If attempts to reach Examiner by telephone are unsuccessful, Examiner’s supervisor, Wei Mui, can be reached at (571) 272-3708. Information regarding the status of an application may be obtained from the Patent Center system. For more information about the Patent Center system, see https://www.uspto.gov/patents/apply/patent-center. If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call (800) 786-9199 (in USA or Canada) or (571) 272-1000.
/Andrew M. Lyons/Primary Examiner, Art Unit 2191
Prosecution Insights