DETAILED ACTION
This action is in response to the claims filed February 12, 2024. Claims 1-20 are pending. Claims 1, 9, and 16 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 14 and 15 are objected to because of the following informalities:
- Claim 14 recites “whether the one or more test conditions are comprises:”. This should likely read “whether the one or more test conditions are validated comprises”.
- Claim 15 recites “whether the one or more test conditions are comprises:”. This should likely read “whether the one or more test conditions are validated comprises”.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, the limitations “detect input data to a component of an application, the input data having a data type”, “configure, based on the data type, one or more log events that are associated with one or more event indicators”, and “determine, using the subset of application log data, whether one or more test conditions are validated” as drafted, are functions that, under their broadest reasonable interpretation, recite the abstract idea of a mental process. The limitation encompasses a human mind carrying out the function through observation, evaluation, judgement, and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and call under the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The additional elements “A system for component testing using log events, the system comprising: one or more memories; and one or more processors, communicatively coupled to the one or more memories, configured to”, “execute the component of the application”, and “perform one or more actions based on whether the one or more test conditions are validated” are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer, and/or mere computer components. See MPEP 2106.05(f). The additional limitation “obtain, based on the execution of the component, a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the component of the application” do nothing more than add the insignificant extra solution activity of merely gathering and transmitting data to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claim is therefore directed to the judicial exception.
Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of “A system for component testing using log events, the system comprising: one or more memories; and one or more processors, communicatively coupled to the one or more memories, configured to”, “execute the component of the application”, and “perform one or more actions based on whether the one or more test conditions are validated” amount to no more than mere instructions, or generic computer/computer components to carry out the exception. See MPEP 2106.05(f). For the limitations “obtain, based on the execution of the component, a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the component of the application”, the courts have identified mere data gathering and transmitting to be well-understood, routine, and conventional. See MPEP 2106.05(d). Accordingly, the claims are not patent eligible under 35 U.S.C. 101.
Claim 2 does not recite additional mental steps. The limitation “wherein the one or more processors, to perform the one or more actions, are configured to: perform the one or more actions during the execution of the component and in association with a function of the application” amount to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 3, the limitations “to determine whether the one or more test conditions are validated”, “detect, based on the subset of application log data, that an exception event for a function of the application has occurred”, and “determined, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring” are additional mental steps. The limitation “wherein the one or more processors…are configured to” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 4, the limitation “based on determining that the retry was not performed based on the exception event occurring” is an additional mental step. The limitations “wherein the one or more processors, to perform the one or more actions are configured to” and “cause the retry for the function to be performed” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 5, the limitation “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” is an additional mental step. The limitations “wherein the one or more processors, to perform the one or more actions, are configured to” and “cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 6, the limitation “to determine whether the one or more test conditions are validated”, and “determine whether the subset of application log data indicates that the one or more rules were applied” are additional mental steps. The limitation “wherein a log event, of the one or more log events, is associated with one or more rules being applied by the component during the execution flow” merely further describes the “log events” of the mental steps of claim 1, and thus amounts to a mental step and cannot amount to practical application under Prong 2, nor to significantly more under Step 2B. The limitation “wherein the one or more processors…are configured to” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 7, the limitation “to determine whether the one or more test conditions are validated”, “detect, based on the subset of application log data, that the exception event occurred”, and “determine, based on detecting that the exception event occurred, whether the subset of application log data indicates that one or more exception event handling operations were performed” are additional mental steps. The limitation “wherein a log event, of the one or more log events, is associated with an exception event” merely further describes the “log events” of the mental steps of claim 1, and thus amounts to a mental step and cannot amount to practical application under Prong 2, nor to significantly more under Step 2B. The limitation “wherein the one or more processors…are configured to” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Claim 8 does not recite additional mental steps. The limitation “wherein the one or more event indicators include at least one of: a keyword, a flag, or an event marker” merely further describes the event indicators of the mental steps of claim 1, and thus amounts to a mental step under Prong 1, which cannot amount to practical application under Prong 2, nor to significantly more under Step 2B.
Regarding claim 9, the limitations “determining, …and based on the data type, one or more log events that are indicative of one or more test conditions for the component and the data type” and “determining, by the device and using the subset of application log data, whether the one or more test conditions are validated” as drafted, are functions that, under their broadest reasonable interpretation, recite the abstract idea of a mental process. The limitation encompasses a human mind carrying out the function through observation, evaluation, judgement, and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and call under the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The additional elements “…by a device” and “performing, by the device, one or more actions based on whether the one or more test conditions are validated” are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer, and/or mere computer components. See MPEP 2106.05(f). The additional limitations “obtaining,…, input data to a component of an application, the input data having a data type” and “obtaining… and based on an execution of the component, a subset of application log data, from a set of application log data, that is associated with the one or more log events” do nothing more than add the insignificant extra solution activity of merely gathering and transmitting data to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claim is therefore directed to the judicial exception.
Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of “…by a device” and “performing, by the device, one or more actions based on whether the one or more test conditions are validated” amount to no more than mere instructions, or generic computer/computer components to carry out the exception. See MPEP 2106.05(f). For the limitations “obtaining,…, input data to a component of an application, the input data having a data type” and “obtaining… and based on an execution of the component, a subset of application log data, from a set of application log data, that is associated with the one or more log events”, the courts have identified mere data gathering and transmitting to be well-understood, routine, and conventional. See MPEP 2106.05(d). Accordingly, the claims are not patent eligible under 35 U.S.C. 101.
Claim 10 does not recite additional mental steps. The limitation “wherein obtaining the subset of application log data comprises: configuring, for the component, a lot data extraction component to collect application log data that includes event indicators that are indicative of the one or more log event” amounts to mere data gathering, which is insignificant extra solution activity which does not amount to practical application under Prong 2, nor to significantly more under Step 2B.
Claim 11 does not recite additional mental steps. The limitation “wherein performing the one or more actions comprises: performing the one or more actions during the execution of the component and in association with a function of the application” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 12, the limitation “wherein determining whether the one or more test conditions are validated comprises”, “detecting, based on the subset of application log data, that an exception event for a function of the application has occurred”, and “determining, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring” are additional mental steps. Claim 12 does not recite additional limitations which integrate the claim into a practical application under Prong 2, nor amount to significantly more under Step 2B.
Regarding claim 13, the limitations “detecting, based on the subset of application log data, a retry event for a function of the application” is an additional mental step. The limitation “wherein performing the one or more actions comprises” and “causing, based on detecting that the retry event, one or more retries for the function to be performed” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 14, the limitations “wherein determining whether the one or more test conditions are comprises” and “determining whether the subset of application log data indicates that an execution flow for a function was performed” are additional mental steps. Claim 14 does not recite additional limitations which integrate the claim into a practical application under Prong 2, nor amount to significantly more under Step 2B.
Regarding claim 15, the limitation “wherein determining whether the one or more test conditions are comprises”, “detecting, based on the subset of application log data, that the exception event occurred”, and “determining, based on detecting that the exception event occurred, whether the subset of application log data indicates that one or more exception event handling operations were performed” are additional mental steps. The limitation “wherein a log event, of the one or more log events, is associated with an exception event” merely further describes the “log events” of the mental steps of claim 1, and thus amounts to a mental step and cannot amount to practical application under Prong 2, nor to significantly more under Step 2B.
Regarding claim 16, the limitations “detect input data to a component of an application, the input data having a data type”, “configure, based on the data type, one or more log events that are associated with one or more event indicators”, and “validate, using the subset of application log data, one or more test conditions” as drafted, are functions that, under their broadest reasonable interpretation, recite the abstract idea of a mental process. The limitation encompasses a human mind carrying out the function through observation, evaluation, judgement, and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and call under the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The additional elements “A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to” are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer, and/or mere computer components. See MPEP 2106.05(f). The additional limitation “collect, based on the execution of the component, a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the component of the application” does nothing more than add the insignificant extra solution activity of merely gathering and transmitting data to the judicial exception. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claim is therefore directed to the judicial exception.
Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to” amount to no more than mere instructions, or generic computer/computer components to carry out the exception. See MPEP 2106.05(f). For the limitations “collect, based on the execution of the component, a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the component of the application”, the courts have identified mere data gathering and transmitting to be well-understood, routine, and conventional. See MPEP 2106.05(d). Accordingly, the claims are not patent eligible under 35 U.S.C. 101.
Regarding claim 17, the limitations “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” and “determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring” are additional mental steps. The limitation “wherein the one or more instructions, that cause the device to validate the one or more test conditions, cause the device to” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 18, the limitation “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” is an additional mental step. The limitations “wherein the one or more instructions further cause the device to:” and “cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 19, the limitation “determine whether the subset of application log data indicates that the one or more rules were applied” is an additional mental step. The limitation “wherein a log event, of the one or more log events, is associated with one or more rules being applied by the component during execution flow” merely further describes the “log events” of the mental steps of claim 1, and thus amounts to a mental step and cannot amount to practical application under Prong 2, nor to significantly more under Step 2B. The limitation “wherein the one or more processors, to validate the one or more test conditions, are configured to” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claim 20 the limitations “detect, based on the subset of application log data, that the exception event occurred” and “determine, based on detecting that the exception event occurred, whether the subset of application log data indicates that one or more exception event handling operations were performed” are additional mental steps. The limitations “wherein a log event, of the one or more log events, is associated with an exception event” merely further describes the “log events” of the mental steps of claim 1, and thus amounts to a mental step and cannot amount to practical application under Prong 2, nor to significantly more under Step 2B. The limitation “wherein the one or more processors, to validate the one or more test conditions, are configured to:” amounts to the mere instruction to apply a generic computer/computer components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-14, and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over US 20140068567 A1 (hereinafter “Smith”), further in view of US 6029257 A (hereinafter “Palmer”), and further in view of “Testability of Software Components” by Freedman (hereinafter “Freedman”).
Regarding claim 1, Smith discloses:
A system for [source code] testing using log events, the system comprising (Paragraph [0016]; Paragraph [0003], “In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
one or more memories (Paragraph [0016]); and
one or more processors, communicatively coupled to the one or more memories, configured to (Paragraph [0016]):
- …
- configure… one or more log events that are associated with one or more event indicators (Paragraph [0005], “In embodiments, the state machine is extensible to add additional types of the events that are utilized to determine the relevant events associated with the defect in the source code. In implementations, the events can be generated for different types of the events, including: an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code; a declaration event that indicates an operation on a variable; and a failed-assertion event that indicates the defect in the source code”) [Examiner’s remarks: One or more log events are configured which indicate events, including for example, branch events and failed-assertion events.];
- execute the [source code] of the application (Paragraph [0003], “In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions”);
- obtain, based on the execution of the [source code], a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the [source code] of the application (Paragraphs [0003]-[0004], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code [obtain, based on the execution of the [source code]]. A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code [a subset of application log data form a set of application log data]. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect. In embodiments, the state machine is also implemented to discard non-relevant events of the logged events, as well as determine related relevant events associated with the defect in the source code, and merge the related relevant events as a single relevant event. Not all of the relevant events may contribute to the defect in the source code, and the state machine can be implemented filter out redundant relevant events and determine which of the relevant events contribute to a duplicate defect in the source code. In embodiments, each of the relevant events has an associated tag that indicates at least a type of the event, an importance of the event, and a message to explain the event [wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the [source code] of the application]”; Paragraph [0005], “In implementations, the events can be generated for different types of the events, … a branch event that indicates a change of executable paths in the source code; …”) [Examiner’s remarks: From the execution, application data may be collected along the path of execution indicating the execution flow of the code. A subset of that application data may be retrieved through the state machine traversal which include event indicators which indicate the type of event that occurred and its relevancy to an error.];
- determine, using the subset of application log data, whether one or more test conditions are validated (Paragraphs [0018]-[0019], “In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code. …The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code. …The source code analysis 106 generates the events 110, which can include the relevant events 116 that are associated with a defect in the source code, and the non-relevant events 118 are discarded”) [Examiner’s remarks: Based on the one or more logged events, it can be determined whether some test conditions are valid, for example, through an error or through a failed-assertion event.]; and
- perform one or more actions based on whether the one or more test conditions are validated (Paragraph [0003], “A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect”) [Examiner’s remarks: Based on the test condition not being valid (error), actions including analysis of the error and reporting of relevant events is performed.].
Smith discloses testing [source code]. Smith does not explicitly disclose the source code being tested in components. However, Palmer discloses components (Column 1, line 63 – Column 2, line 2, “In accordance with one aspect of the invention, a method, apparatus, and computer program product for testing a computer system having a set of computer components (hardware and/or software) and a testing program stored in memory provides both a test header file associated with a selected component, and a component file listing the set of components of the computer system”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “component”. As stated in Palmer, “Computer systems typically include a plurality of hard-ware and software components (“components”) that are to be tested” (Column 1, lines 34-36). Component based programing is increasingly common for it’s provided versatility. Having efficient ways to test their correctness is thus, also increasingly important. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing of software with that software being in component form.
The combination of Smith and Palmer does not explicitly disclose:
- detect input data to a component of an application, the input data having a data type;
- …based on the data type…
However, Freedman discloses:
- detect input data to a component of an application, the input data having a data type (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”) [Examiner’s remarks: Freedman discloses inputs to a component of an application, which have specific data types for the purposes of testing the components domain. One of ordinary skill in the art may combine the testing of input data with specific data types with the component testing of Smith and Palmer.];
- … based on the data type… (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Freedman into the combined teachings of Smith and Palmer to include “detect input data to a component of an application, the input data having a data type” and “based on the data type”. As stated in Freedman, “Testing is used to assess the compliance of a program to its intended specifications and to assess the reliability of the program to inputs which were not intended to be part of the specifications. Good test inputs are those that result in a high probability of discovering errors upon program execution” (Page 553). Freedman belongs to the analogous art of component testing. Understanding the types of input allowed and needed for testing a component is necessary for testing the full scope of the component. Detecting data type allows testers to know what condition was tested. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with detection of data types.
Regarding claim 2, the rejection of claim 1 is incorporated; and Smith does not explicitly disclose:
- perform the one or more actions during the execution of the component and in association with a function of the application.
However, Palmer discloses:
- perform the one or more actions during the execution of the component and in association with a function of the application (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: An action (retrying the test) is performed, which executes the component (during execution of the component) and is in associated with the function of the component (testing the function of the component).].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “perform the one or more actions during the execution of the component and in association with a function of the application”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Automated logging, retrying, and performing other actions in response to a confirmation or not of validity of a test simplifies the process of application testing, by reducing the possibility of human intervention and error. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating further events.
Regarding claim 3, the rejection of claim 1 is incorporated; and Smith further discloses:
wherein the one or more processors, to determine whether the one or more test conditions are validated, are configured to (Paragraph [0003], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
Smith does not explicitly disclose:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred; and
- determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring.
However, Palmer discloses:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred (Column 7, lines 5-9, “Specifically, when the exit code in this example equals zero, then the test is considered to have passed. When the exit code equals one, then the test is considered to have failed. The numeric exit code, indication of whether it is a pass or fail code, and the message string at the end of the lines will be included in a log file if the exit code is matched”; Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: An application data log in the header can be used to detect a failure occurring.]; and
- determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: Based on the log data of the header file, it can be determined if a test was retried based on the number of retries.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” and “determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 4, the rejection of claim 3 is incorporated; and Smith does not explicitly disclose:
- cause the retry for the function to be performed based on determining that the retry was not performed based on the exception event occurring.
However, Palmer discloses:
- cause the retry for the function to be performed based on determining that the retry was not performed based on the exception event occurring (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: Based on the number of retries not reaching a limit (which may be 1), the function is retried again.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “cause the retry for the function to be performed based on determining that the retry was not performed based on the exception event occurring”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 5, the rejection of claim 1 is incorporated; and Smith does not explicitly disclose:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred; and
- cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed.
However, Palmer discloses:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred (Column 7, lines 5-9, “Specifically, when the exit code in this example equals zero, then the test is considered to have passed. When the exit code equals one, then the test is considered to have failed. The numeric exit code, indication of whether it is a pass or fail code, and the message string at the end of the lines will be included in a log file if the exit code is matched”; Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”); and
- cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: One or more retries may be performed based on the test failing.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” and “cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 6, the rejection of claim 1 is incorporated, and Smith further discloses:
- wherein a log event, of the one or more log events, is associated with one or more rules being applied by the component during the execution flow, and wherein the one or more processors, to determine whether the one or more test conditions are validated, are configured to (Paragraph [0020], “The redundant relevant events may occur as branch events and can be determined from duplicate defects, such as on alternate analysis paths in the source code. For example, a first redundant relevant event 120 may be associated with a defect 108 as determined along an executable path in the source code 102, and a second redundant relevant event is associated with the defect as determined along a different executable path in the source code. For example, if a defect occurs whether a branch of the source code is analyzed or skipped, then it is likely that events generated from an analysis of the branch will not provide or contribute to a user understanding of a defect (i.e., although the branch is on the path, and relevant, it does not provide the user useful information because the defect occurs on either side of the branch). Although a branch event may be relevant to the analysis, it can be determined as redundant because the analysis would have reached the same result if a different branch decision is selected for analysis”) [Examiner’s remarks: Some log events of the plurality of log events may indicate branch events associated with different branches on an execution path, which are reached through different applied rules.]:
- determine whether the subset of application log data indicates that the one or more rules were applied (Paragraph [0018], “The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes)”; Paragraph [0020], “The redundant relevant events may occur as branch events and can be determined from duplicate defects, such as on alternate analysis paths in the source code. For example, a first redundant relevant event 120 may be associated with a defect 108 as determined along an executable path in the source code 102, and a second redundant relevant event is associated with the defect as determined along a different executable path in the source code. For example, if a defect occurs whether a branch of the source code is analyzed or skipped, then it is likely that events generated from an analysis of the branch will not provide or contribute to a user understanding of a defect (i.e., although the branch is on the path, and relevant, it does not provide the user useful information because the defect occurs on either side of the branch). Although a branch event may be relevant to the analysis, it can be determined as redundant because the analysis would have reached the same result if a different branch decision is selected for analysis”) [Examiner’s remarks: A branch event may indicate that control flow has changed at a branch, indicating an application of a rule at the branch.].
Regarding claim 8, the rejection of claim 1 is incorporated; and Smith further discloses:
wherein the one or more event indicators include at least one of:
- a keyword, a flag, or an event marker (Paragraph [0004], “ In embodiments, each of the relevant events has an associated tag that indicates at least a type of the event, an importance of the event, and a message to explain the event”).
Regarding claim 9, Smith discloses:
A method for [source code] testing using log events, comprising (Paragraph [0016]; Paragraph [0003], “In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
…
- determining, by the device…, one or more log events that are indicative of one or more test conditions for the [source code]… (Paragraph [0005], “In embodiments, the state machine is extensible to add additional types of the events that are utilized to determine the relevant events associated with the defect in the source code. In implementations, the events can be generated for different types of the events, including: an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code; a declaration event that indicates an operation on a variable; and a failed-assertion event that indicates the defect in the source code”) [Examiner’s remarks: One or more log events are determined which indicate events, including for example, branch events and failed-assertion events. Failed assertion events may be considered indicative of a test condition of a source code.];
- obtaining, by the device and based on an execution of the [source code], a subset of application log data, from a set of application log data, that is associated with the one or more log events (Paragraphs [0003]-[0004], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code [obtain, by the device and based on the execution of the [source code]]. A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code [a subset of application log data from a set of application log data]. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect. In embodiments, the state machine is also implemented to discard non-relevant events of the logged events, as well as determine related relevant events associated with the defect in the source code, and merge the related relevant events as a single relevant event. Not all of the relevant events may contribute to the defect in the source code, and the state machine can be implemented filter out redundant relevant events and determine which of the relevant events contribute to a duplicate defect in the source code. In embodiments, each of the relevant events has an associated tag that indicates at least a type of the event, an importance of the event, and a message to explain the event [that is associated with the one or more log events]”) [Examiner’s remarks: From the execution, application data may be collected along the path of execution indicating the execution flow of the code. A subset of that application data may be retrieved through the state machine traversal which include event indicators which indicate the type of event that occurred and its relevancy to an error.];
- determining, by the device and using the subset of application log data, whether the one or more test conditions are validated (Paragraphs [0018]-[0019], “In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code. …The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code. …The source code analysis 106 generates the events 110, which can include the relevant events 116 that are associated with a defect in the source code, and the non-relevant events 118 are discarded”) [Examiner’s remarks: Based on the one or more logged events, it can be determined whether some test conditions are valid, for example, through an error or through a failed-assertion event.]; and
- performing, by the device, one or more actions based on whether the one or more test conditions are validated (Paragraph [0003], “A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect”) [Examiner’s remarks: Based on the test condition not being valid (error), actions including analysis of the error and reporting of relevant events is performed.].
Smith discloses testing [source code]. Smith does not explicitly disclose the source code being tested in components. However, Palmer discloses components (Column 1, line 63 – Column 2, line 2, “In accordance with one aspect of the invention, a method, apparatus, and computer program product for testing a computer system having a set of computer components (hardware and/or software) and a testing program stored in memory provides both a test header file associated with a selected component, and a component file listing the set of components of the computer system”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “component”. As stated in Palmer, “Computer systems typically include a plurality of hard-ware and software components (“components”) that are to be tested” (Column 1, lines 34-36). Component based programing is increasingly common for its provided versatility. Having efficient ways to test their correctness is thus, also increasingly important. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing of software with that software being in component form.
The combination of Smith and Palmer does not explicitly disclose:
- obtaining, by a device, input data to a component of an application, the input data having a data type;
- … and based on the data type…and the data type;
However, Freedman discloses:
- obtaining, by a device, input data to a component of an application, the input data having a data type (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”) [Examiner’s remarks: Freedman discloses inputs to a component of an application, which have specific data types for the purposes of testing the components domain. One of ordinary skill in the art may combine the testing of input data with specific data types with the component testing of Smith and Palmer.];
- …and based on the data type…and the data type (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”) [Examiner’s remarks: The testing can be configured based on input data types.];
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Freedman into the combined teachings of Smith and Palmer to include “obtaining, by a device, input data to a component of an application, the input data having a data type” and “… and based on the data type…and the data type”. As stated in Freedman, “Testing is used to assess the compliance of a program to its intended specifications and to assess the reliability of the program to inputs which were not intended to be part of the specifications. Good test inputs are those that result in a high probability of discovering errors upon program execution” (Page 553). Freedman belongs to the analogous art of component testing. Understanding the types of input allowed and needed for testing a component is necessary for testing the full scope of the component. Detecting data type allows testers to know what condition was tested. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with detection of data types.
Regarding claim 10, the rejection of claim 9 is incorporated; and Smith further discloses:
wherein obtaining the subset of application log data comprises:
- configuring, for the component, a log data extraction component to collect application log data that includes event indicators that are indicative of the one or more log events (Paragraph [0003]-[0004], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code. A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect. In embodiments, the state machine is also implemented to discard non-relevant events of the logged events, as well as determine related relevant events associated with the defect in the source code, and merge the related relevant events as a single relevant event. Not all of the relevant events may contribute to the defect in the source code, and the state machine can be implemented filter out redundant relevant events and determine which of the relevant events contribute to a duplicate defect in the source code. In embodiments, each of the relevant events has an associated tag that indicates at least a type of the event, an importance of the event, and a message to explain the event”) [Examiner’s remarks: A log data extraction component (key event manager) collects log events along executable paths in code.].
Regarding claim 11, the rejection of claim 9 is incorporated; and Smith does not explicitly disclose:
- performing the one or more actions during the execution of the component and in association with a function of the application.
However, Palmer discloses:
- performing the one or more actions during the execution of the component and in association with a function of the application (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: An action (retrying the test) is performed, which executes the component (during execution of the component) and is in associated with the function of the component (testing the function of the component).].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “performing the one or more actions during the execution of the component and in association with a function of the application”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Automated logging, retrying, and performing other actions in response to a confirmation or not of validity of a test simplifies the process of application testing, by reducing the possibility of human intervention and error. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating further events.
Regarding claim 12, the rejection of claim 11 is incorporated; and Smith discloses:
wherein determining whether the one or more test conditions are validated comprises (Paragraph [0003], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
Smith does not explicitly disclose:
- detecting, based on the subset of application log data, that an exception event for a function of the application has occurred; and
- determining, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring.
However, Palmer discloses:
- detecting, based on the subset of application log data, that an exception event for a function of the application has occurred (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: An application data log in the header can be used to detect a failure occurring.]; and
- determining, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: Based on the log data of the header file, it can be determined if a test was retried based on the number of retries.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detecting, based on the subset of application log data, that an exception event for a function of the application has occurred” and “determining, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring.”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 13, the rejection of claim 9 is incorporated; and Smith does not explicitly disclose:
- detecting, based on the subset of application log data, a retry event for a function of the application; and
- causing, based on detecting that the retry event, one or more retries for the function to be performed.
However, Palmer discloses:
- detecting, based on the subset of application log data, a retry event for a function of the application (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”); and
- causing, based on detecting that the retry event, one or more retries for the function to be performed (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: One or more retries may be performed based on the test failing.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detecting, based on the subset of application log data, a retry event for a function of the application” and “causing, based on detecting that the retry event, one or more retries for the function to be performed”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 14, the rejection of claim 9 is incorporated; and Smith further discloses:
wherein determining whether the one or more test conditions are comprises (Paragraph [0003], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
- determining whether the subset of application log data indicates that an execution flow for a function was performed (Paragraph [0005], “In embodiments, the state machine is extensible to add additional types of the events that are utilized to determine the relevant events associated with the defect in the source code. In implementations, the events can be generated for different types of the events, including: an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code; a declaration event that indicates an operation on a variable; and a failed-assertion event that indicates the defect in the source code”) [Examiner’s remarks: The log data can be looked at to determine, for example, a branch event which indicated execution flow.].
Regarding claim 16, Smith discloses:
A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising (Paragraph [0016]):
one or more instructions that, when executed by one or more processors of a device, cause the device to (Paragraph [0016]):
- …
- configure, …, one or more log events that are associated with one or more event indicators (Paragraph [0005], “In embodiments, the state machine is extensible to add additional types of the events that are utilized to determine the relevant events associated with the defect in the source code. In implementations, the events can be generated for different types of the events, including: an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code; a declaration event that indicates an operation on a variable; and a failed-assertion event that indicates the defect in the source code”) [Examiner’s remarks: One or more log events are configured which indicate events, including for example, branch events and failed-assertion events.];
- collect, based on the execution of the [source code], a subset of application log data from a set of application log data, wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the [source code] of the application (Paragraphs [0003]-[0004], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code [collect, based on the execution of the [source code]]. A state machine is implemented to traverse back through the logged events and determine relevant events that are associated with the defect in the source code [a subset of application log data form a set of application log data]. An analysis output of the relevant events can then be generated to indicate the defect in the source code and a cause of the defect. In embodiments, the state machine is also implemented to discard non-relevant events of the logged events, as well as determine related relevant events associated with the defect in the source code, and merge the related relevant events as a single relevant event. Not all of the relevant events may contribute to the defect in the source code, and the state machine can be implemented filter out redundant relevant events and determine which of the relevant events contribute to a duplicate defect in the source code. In embodiments, each of the relevant events has an associated tag that indicates at least a type of the event, an importance of the event, and a message to explain the event [wherein the subset of application log data includes at least one of the one or more event indicators, and wherein the set of application log data includes information indicative of an execution flow for the [source code] of the application]”; Paragraph [0005], “In implementations, the events can be generated for different types of the events, … a branch event that indicates a change of executable paths in the source code; …”) [Examiner’s remarks: From the execution, application data may be collected along the path of execution indicating the execution flow of the code. A subset of that application data may be retrieved through the state machine traversal which include event indicators which indicate the type of event that occurred and its relevancy to an error.]; and
- validate, using the subset of application log data, one or more test conditions (Paragraphs [0018]-[0019], “In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code. …The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code. …The source code analysis 106 generates the events 110, which can include the relevant events 116 that are associated with a defect in the source code, and the non-relevant events 118 are discarded”) [Examiner’s remarks: Based on the one or more logged events, it can be determined whether some test conditions are valid, for example, through an error or through a failed-assertion event.].
Smith discloses testing [source code]. Smith does not explicitly disclose the source code being tested in components. However, Palmer discloses components (Column 1, line 63 – Column 2, line 2, “In accordance with one aspect of the invention, a method, apparatus, and computer program product for testing a computer system having a set of computer components (hardware and/or software) and a testing program stored in memory provides both a test header file associated with a selected component, and a component file listing the set of components of the computer system”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “component”. As stated in Palmer, “Computer systems typically include a plurality of hard-ware and software components (“components”) that are to be tested” (Column 1, lines 34-36). Component based programing is increasingly common for its provided versatility. Having efficient ways to test their correctness is thus, also increasingly important. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing of software with that software being in component form.
The combination of Smith and Palmer does not explicitly disclose:
- detect input data to a component of an application, the input data having a data type;
- …based on the data type
However, Freedman discloses:
- detect input data to a component of an application, the input data having a data type (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”) [Examiner’s remarks: Freedman discloses inputs to a component of an application, which have specific data types for the purposes of testing the components domain. One of ordinary skill in the art may combine the testing of input data with specific data types with the component testing of Smith and Palmer.];
- …based on the data type (Page 553, “Before testing begins, it is necessary to identify the input and output data characteristics (i.e., type, format, range) of the component and the "states" of the component. During testing, the input-output data characteristics may be inferred from specific identifiers in the program; states and state transitions usually are not associated with specific identifiers”);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Freedman into the combined teachings of Smith and Palmer to include “detect input data to a component of an application, the input data having a data type” and “based on the data type”. As stated in Freedman, “Testing is used to assess the compliance of a program to its intended specifications and to assess the reliability of the program to inputs which were not intended to be part of the specifications. Good test inputs are those that result in a high probability of discovering errors upon program execution” (Page 553). Freedman belongs to the analogous art of component testing. Understanding the types of input allowed and needed for testing a component is necessary for testing the full scope of the component. Detecting data type allows testers to know what condition was tested. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with detection of data types.
Regarding claim 17, the rejection of claim 16 is incorporated; and Smither further discloses:
wherein the one or more instructions, that cause the device to validate the one or more test conditions, cause the device to (Paragraph [0003], “Determining relevant events in source code analysis are described. In embodiments, a computing device includes a key event manager that is implemented to traverse executable paths in source code of executable software instructions, log events along the executable paths in the source code, and determine a defect in the source code along an executable path in the source code”):
Smith does not explicitly disclose:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred; and
- determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring.
However, Palmer discloses:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: An application data log in the header can be used to detect a failure occurring.]; and
- determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: Based on the log data of the header file, it can be determined if a test was retried based on the number of retries.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” and “determine, based on the subset of application log data, whether a retry for the function was performed based on the exception event occurring”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 18, the rejection of claim 16 is incorporated; and Smith does not explicitly disclose:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred; and
- cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed.
However, Palmer discloses:
- detect, based on the subset of application log data, that an exception event for a function of the application has occurred (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”); and
- cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed (Column 10, lines 2-13, “Conversely, if the component is considered to have failed the test, then the process continues to step 424 in which it is determined if the test may be retried. As noted above, the number of retries is listed in the test header file 300. If the test may not be retried, or the total allotted number of retries has been expended, then the process ends. If the test may be retried, however, then an error message is displayed on the controller GUI 600 (step 426) and the process loops back to step 416 to determine if the AutoStart variable is set. This process continues until each test object 202 in the subset of test objects 202 is utilized”) [Examiner’s remarks: One or more retries may be performed based on the test failing.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Palmer into the teachings of Smith to include “detect, based on the subset of application log data, that an exception event for a function of the application has occurred” and “cause, based on detecting that the exception event has occurred, one or more retries for the function to be performed”. As stated in Palmer, “In addition to confirming the operability of computer systems, it also is desirable for data from computer hardware tests to be logged for quality control purposes” (Column 1, lines 52-54). Palmer belongs to the analogous art of component testing using log events. Automating the retrying of component tests allows automation of testing without human interference. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing to automating log and retries of tests.
Regarding claim 19, the rejection of claim 16 is incorporated; and Smith further discloses:
wherein a log event, of the one or more log events, is associated with one or more rules being applied by the component during the execution flow, and wherein the one or more processors, to validate the one or more test conditions, are configured to (Paragraph [0020], “The redundant relevant events may occur as branch events and can be determined from duplicate defects, such as on alternate analysis paths in the source code. For example, a first redundant relevant event 120 may be associated with a defect 108 as determined along an executable path in the source code 102, and a second redundant relevant event is associated with the defect as determined along a different executable path in the source code. For example, if a defect occurs whether a branch of the source code is analyzed or skipped, then it is likely that events generated from an analysis of the branch will not provide or contribute to a user understanding of a defect (i.e., although the branch is on the path, and relevant, it does not provide the user useful information because the defect occurs on either side of the branch). Although a branch event may be relevant to the analysis, it can be determined as redundant because the analysis would have reached the same result if a different branch decision is selected for analysis”) [Examiner’s remarks: Some log events of the plurality of log events may indicate branch events associated with different branches on an execution path, which are reached through different applied rules.]:
- determine whether the subset of application log data indicates that the one or more rules were applied (Paragraph [0020], “The redundant relevant events may occur as branch events and can be determined from duplicate defects, such as on alternate analysis paths in the source code. For example, a first redundant relevant event 120 may be associated with a defect 108 as determined along an executable path in the source code 102, and a second redundant relevant event is associated with the defect as determined along a different executable path in the source code. For example, if a defect occurs whether a branch of the source code is analyzed or skipped, then it is likely that events generated from an analysis of the branch will not provide or contribute to a user understanding of a defect (i.e., although the branch is on the path, and relevant, it does not provide the user useful information because the defect occurs on either side of the branch). Although a branch event may be relevant to the analysis, it can be determined as redundant because the analysis would have reached the same result if a different branch decision is selected for analysis”) [Examiner’s remarks: A branch event may indicate that control flow has changed at a branch, indicating an application of a rule at the branch.].
Claims 7, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20140068567 A1 (hereinafter “Smith”), further in view of US 6029257 A (hereinafter “Palmer”), and further in view of “Testability of Software Components” by Freedman (hereinafter “Freedman”), and further in view of “Event Logs for the Analysis of Software Failures: A Rule Based Approach” by Cinque et. al (hereinafter “Cinque”).
Regarding claim 7, the rejection of claim 1 is incorporated; and Smith further discloses:
wherein a log event, of the one or more log events, is associated with an exception event, and wherein the one or more processors, to determine whether the one or more test conditions are validated, are configured to (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”):
- detect, based on the subset of application log data, that the exception event occurred (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”) [Examiner’s remarks: An exception event is detected (failed-assertion event).]; and
- determine, based on detecting that the exception event occurred (Paragraph [0019], “The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code”) [Examiner’s remark: Based on determining that an exception occurred, action is taken to explore the log events.]…
The combination of Smith, Palmer, and Freedman does not explicitly disclose:
- …whether the subset of application log data indicates that one or more exception event handling operations were performed.
However, Cinque discloses:
- …whether the subset of application log data indicates that one or more exception event handling operations were performed (Page 809, “try{...} catch(...) {log_error()}. The pattern was observed in 12 percent of cases (catch category). The logging function is invoked within the catch block once an exception is raised during the execution of the try block.”) [Examiner’s remarks: Cinque discloses log data that logs try catches, a type of error handling. One of ordinary skill in the art may combine with action of Smith and the exception of Cinque.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Cinque into the combined teachings of Smith, Palmer, and Freedman to include “whether the subset of application log data indicates that one or more exception event handling operations were performed”. As stated in Cinque, “Logs are used for a variety of purposes, such as debugging, configuration handling, access control, monitoring, and profiling. Moreover, logs are often the only means to analyze the failure behavior of the system under real workload conditions” (Page 806). Cinque belongs to the analogous art of component testing using log events. Logging is useful in debugging and other software development processes, and logging error handling events allows developers to see errors that may not otherwise have been visible, or to analyze certain control flow structures. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with exception event logging.
Regarding claim 15, the rejection of claim 9 is incorporated; and Smith further discloses:
wherein a log event, of the one or more log events, is associated with an exception event, and wherein determining whether the one or more test conditions are comprises (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”):
- detecting, based on the subset of application log data, that the exception event occurred (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”) [Examiner’s remarks: An exception event is detected (failed-assertion event).]; and
- determining, based on detecting that the exception event occurred (Paragraph [0019], “The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code”) [Examiner’s remark: Based on determining that an exception occurred, action is taken to explore the log events.]…
The combination of Smith, Palmer, and Freedman does not explicitly disclose:
- … whether the subset of application log data indicates that one or more exception event handling operations were performed.
However, Cinque discloses:
- …whether the subset of application log data indicates that one or more exception event handling operations were performed (Page 809, “try{...} catch(...) {log_error()}. The pattern was observed in 12 percent of cases (catch category). The logging function is invoked within the catch block once an exception is raised during the execution of the try block.”) [Examiner’s remarks: Cinque discloses log data that logs try catches, a type of error handling. One of ordinary skill in the art may combine with action of Smith and the exception of Cinque.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Cinque into the combined teachings of Smith, Palmer, and Freedman to include “whether the subset of application log data indicates that one or more exception event handling operations were performed”. As stated in Cinque, “Logs are used for a variety of purposes, such as debugging, configuration handling, access control, monitoring, and profiling. Moreover, logs are often the only means to analyze the failure behavior of the system under real workload conditions” (Page 806). Cinque belongs to the analogous art of component testing using log events. Logging is useful in debugging and other software development processes, and logging error handling events allows developers to see errors that may not otherwise have been visible, or to analyze certain control flow structures. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with exception event logging.
Regarding claim 20, the rejection of claim 16 is incorporated; and Smith further discloses:
wherein a log event, of the one or more log events, is associated with an exception event, and wherein the one or more processors, to validate the one or more test conditions, are configured to (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”):
- detect, based on the subset of application log data, that the exception event occurred (Paragraph [0018], “The key event manager 104 is implemented to log events 110 that occur during the static code analysis along the executable paths in the source code. In embodiments, the events are logged and used to determine events that are relevant to a user from the analysis. The events 110 can include different event types 112, such as an alias event that indicates at least two variables correspond to a same reference; a branch event that indicates a change of executable paths in the source code 102 (e.g., control flow switch changes); a declaration event that indicates an operation on a variable (e.g., an uninitialized variable, an integer variable value of zero); and a failed-assertion event that indicates a determined defect in the source code”) [Examiner’s remarks: An exception event is detected (failed-assertion event).]; and
- determine, based on detecting that the exception event occurred (Paragraph [0019], “The key event manager 104 includes a state machine 114 that can be initiated to traverse back through the logged events 110 and determine relevant events 116 that are associated with a defect 108 (i.e., a failed assertion) in the source code”) [Examiner’s remark: Based on determining that an exception occurred, action is taken to explore the log events.] …
The combination of Smith, Palmer, and Freedman does not explicitly disclose:
- … whether the subset of application log data indicates that one or more exception event handling operations were performed.
However, Cinque discloses:
- … whether the subset of application log data indicates that one or more exception event handling operations were performed (Page 809, “try{...} catch(...) {log_error()}. The pattern was observed in 12 percent of cases (catch category). The logging function is invoked within the catch block once an exception is raised during the execution of the try block.”) [Examiner’s remarks: Cinque discloses log data that logs try catches, a type of error handling. One of ordinary skill in the art may combine with action of Smith and the exception of Cinque.].
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Cinque into the combined teachings of Smith, Palmer, and Freedman to include “whether the subset of application log data indicates that one or more exception event handling operations were performed”. As stated in Cinque, “Logs are used for a variety of purposes, such as debugging, configuration handling, access control, monitoring, and profiling. Moreover, logs are often the only means to analyze the failure behavior of the system under real workload conditions” (Page 806). Cinque belongs to the analogous art of component testing using log events. Logging is useful in debugging and other software development processes, and logging error handling events allows developers to see errors that may not otherwise have been visible, or to analyze certain control flow structures. Therefore, it would be obvious to one of ordinary skill in the art to combine log based component testing with exception event logging.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
- “Detecting of Software Failures through Event Logs: an Experimental Study” by Pecchia and Russo discloses using event logs to detect program failures.
- US 20200050534 A1 discloses using event logs to detect system errors.
- “Testing using Log File Analysis: Tools, Methods, and Issues” by Andrews discusses testing of software using log files.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIVIAN WEIJIA DUAN whose telephone number is (703)756-5442. The examiner can normally be reached Monday-Friday 8:30AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Y Mui can be reached at (571) 272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.W.D./Examiner, Art Unit 2191 /WEI Y MUI/Supervisory Patent Examiner, Art Unit 2191