DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made non-final.
This action is in response to the application and claims filed 12 December, 2023. Claims 1-20 are pending in the case and have been examined. Claims -20 are rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
To determine if a claim is directed to patent ineligible subject matter, the Court has guided the Office to apply the Alice/Mayo test, which requires:
Step 1: Determining if the claim falls within a statutory category.
Step 2A: Determining if the claim is directed to a patent ineligible judicial exception consisting of a law of nature, a natural phenomenon, or abstract idea; and Step 2A is a two prong inquiry. MPEP 2106.04(II)(A). Under the first prong, examiners evaluate whether a law of nature, natural phenomenon, or abstract idea is set forth or described in the claim. Abstract ideas include mathematical concepts, certain methods of organizing human activity, and mental processes. MPEP 2104.04(a)(2). The second prong is an inquiry into whether the claim integrates a judicial exception into a practical application. MPEP 2106.04(d).
Step 2B: If the claim is directed to a judicial exception, determining if the claim recites limitations or elements that amount to significantly more than the judicial exception. (See MPEP 2106).
Claims 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-10 are directed to a non-transitory computer-readable storage medium (a manufacture). Claim 11-14 are directed to a computing device comprising one or more processors (a machine), and Claims 15-20 are directed to a method (a process). Therefore, Claims 1-20 are directed to a process, machine or manufacture or composition of matter.
Regarding claim 1
Step 2A Prong 1
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including observation, evaluation, judgement, opinion) or with the aid of pencil and paper but for recitation of generic computer components (e.g., “processor”, “classifier”, and “machine learning model”) [see MPEP 2106.04(a)(2)(III)].
“preprocessing the input data to extract a set of features” (e.g., a human examines data to identify relevant traits, characteristics or attributes in the data set)
“producing a label for each input data to generate a set of labels” (e.g., a human can list, sort, organize data into sets based on similarities between different data and assign a name or category to them)
“constructing a training data set based on an extracted set of features and the set of labels” (e.g., a human can organize the identified traits and assigned labels into a new dataset)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “processor”, “classifier”, and “machine learning model” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)). The Examiner notes that this is used throughout the claim limitations, and is rejected thusly for each claim which recites the same language.
Regarding the “receiving input data relating to software applications in a predetermined format” this additional element is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of inputting data for use in the claimed process (see MPEP 2106.05(g)).
Regarding the “training a machine learning model with the training data set”, and “generating a classifier based on the trained machine learning model” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Regarding the “providing, to the classifier, new input data relating to the software applications” this additional element is recited at a high level of generality and amounts to extra-solution activity of receiving the desired target of the training process, i.e. pre-solution activity of inputting data for use in the claimed process (see MPEP 2106.05(g)).
Regarding the “generating, by the classifier, a predicted label identifying automated tests in response to the new input data”, and “providing the automated tests to a continuous integration and continuous deployment (CI/CD) pipeline” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “processor”, “classifier”, and “machine learning model” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Regarding the “receiving input data relating to software applications in a predetermined format” limitations, this additional element is recited at a high-level of generality and amounts to extra-solution activity of obtaining data to input for a model, i.e., pre-solution activity of data gathering. The courts have found limitations directed to obtaining information electronically, recited at a high-level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding the “training a machine learning model with the training data set”, and “generating a classifier based on the trained machine learning model” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Regarding the “providing, to the classifier, new input data relating to the software applications” limitations, this additional element is recited at a high-level of generality and amounts to extra-solution activity of obtaining data to input for a model, i.e., pre-solution activity of data gathering. The courts have found limitations directed to obtaining information electronically, recited at a high-level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding the “generating, by the classifier, a predicted label identifying automated tests in response to the new input data”, and “providing the automated tests to a continuous integration and continuous deployment (CI/CD) pipeline” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 2
Step 2A Prong 1
Claim 2 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “the generating the classifier further comprises generating a plurality of classifiers”, and “the generating the predicted label further comprises identifying, by the plurality of classifiers, a plurality of attributes of the automated tests, responsive to the new input data relating to the software applications” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “the generating the classifier further comprises generating a plurality of classifiers”, and “the generating the predicted label further comprises identifying, by the plurality of classifiers, a plurality of attributes of the automated tests, responsive to the new input data relating to the software applications” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 3
Step 2A Prong 1
Claim 3 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “identifying, with a first classifier, a first attribute of the automated tests that corresponds to a testsuite to run”, identifying, with a second classifier, a second attribute that corresponds to a service” and “identifying, with a third classifier, a third attribute that corresponds to an action” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “identifying, with a first classifier, a first attribute of the automated tests that corresponds to a testsuite to run”, identifying, with a second classifier, a second attribute that corresponds to a service” and “identifying, with a third classifier, a third attribute that corresponds to an action” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 4
Step 2A Prong 1
Claim 4 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “wherein the operations further comprise performing a postprocessing that concatenates the first attribute, the second attribute and the third attribute into a single prediction” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “wherein the operations further comprise performing a postprocessing that concatenates the first attribute, the second attribute and the third attribute into a single prediction” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 5
Step 2A Prong 1
Claim 5 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “testing the classifier with another set of training data” and “upon testing of the classifier, determining that an accuracy score of the classifier exceeds a predetermined threshold” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “testing the classifier with another set of training data” and “upon testing of the classifier, determining that an accuracy score of the classifier exceeds a predetermined threshold” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 6
Step 2A Prong 1
Claim 6 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including observation, evaluation, judgement, opinion) or with the aid of pencil and paper but for recitation of generic computer components (e.g., “processor”, “classifier”, and “machine learning model”) [see MPEP 2106.04(a)(2)(III)].
“removing one or more stopwords from the extracted set of features” (e.g., a human can evaluate/organize text data and remove predetermined words form it)
“lemmatizing the input data” (e.g., a human can evaluate/organize text data and break them down to their base form, like “walk” form the words “walking”, walked”, and “walks”)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “wherein the input data includes text data”, “the preprocessing the input data further comprises generating reduced text data by: tokenizing the input data into each word” which are recited at a high-level of generality such that they amount to extra-solution activity of defining the data to be used, i.e. pre-solution activity of selecting a particular data source or type of data to be manipulated for use in the claimed process (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “wherein the input data includes text data”, “the preprocessing the input data further comprises generating reduced text data by: tokenizing the input data into each word” limitations, these additional elements are recited at a high-level of generality and amounts to extra-solution activity of pre-solution activity of selecting a particular data source or type of data to be manipulated. The courts have found limitations directed to obtaining information electronically, recited at a high-level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 7
Step 2A Prong 1
Claim 7 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including observation, evaluation, judgement, opinion) or with the aid of pencil and paper but for recitation of generic computer components (e.g., “processor”, “classifier”, and “machine learning model”) [see MPEP 2106.04(a)(2)(III)].
“constructing a second training data set based on different data from the input data” (e.g., a human can organize a second collection of examples using a different data source into a new dataset)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “testing the classifier with the second training data set by comparing a predicted automated test by the classifier with an actual test in response to the second training data set” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)). The Examiner notes that this is used throughout the claim limitations, and is rejected thusly for each claim which recites the same language.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “testing the classifier with the second training data set by comparing a predicted automated test by the classifier with an actual test in response to the second training data set” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 8
Step 2A Prong 1
Claim 8 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “wherein the training the machine learning model with the training data set further comprises training the machine learning model using supervised machine learning by using the extracted set of features as an input and the set of labels associated with the extracted set of features as an output” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “wherein the training the machine learning model with the training data set further comprises training the machine learning model using supervised machine learning by using the extracted set of features as an input and the set of labels associated with the extracted set of features as an output” which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 9
Step 2A Prong 1
Claim 9 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “wherein the operations further comprise generating a notification that the automated tests executed in the CI/CD pipeline have passed or failed” which are recited at a high-level of generality such that they amount to extra-solution activity of generating a notification, i.e. post-solution activity of data outputting for use in the claimed process (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “wherein the operations further comprise generating a notification that the automated tests executed in the CI/CD pipeline have passed or failed” limitations, these additional elements are recited at a high-level of generality and amounts to extra-solution activity of post-solution activity of data outputting. The courts have found limitations directed to obtaining information electronically, recited at a high-level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regarding claim 10
Step 2A Prong 1
Claim 01 does not recite an abstract idea, but is directed to the abstract idea identified in its parents claim(s).
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “wherein the receiving the input data relating to the software applications in the predetermined format further comprises receiving the input data in a text form, an image form, an audio form, a video form, or a combination thereof” which are recited at a high-level of generality such that they amount to extra-solution activity of defining the data to be received, i.e. pre-solution activity of data inputting for use in the claimed process (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “wherein the operations further comprise generating a notification that the automated tests executed in the CI/CD pipeline have passed or failed” limitations, these additional elements are recited at a high-level of generality and amounts to extra-solution activity of pre-solution activity of data inputting. The courts have found limitations directed to obtaining information electronically, recited at a high-level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional element individually or in combination does not amount to significantly more than the judicial exception.
Regrading claims 11-14, which recite substantially the same limitations as claims 1, and 5-8, and are rejected for the same reasons as disclosed above.
Regarding claims 15-20, which recite substantially the same limitations as claims 1-8, and are rejected for the same reasons as disclosed above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8-10, 12, 16, 17, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Raman et al. (US 10073763 B1, referred to as Raman), in view of Nogueira dos Santos et al. (US 20170308790 A1, referred to as Nogueira), in view of Sharma et al. (US 10572374 B2, referred to as Sharma).
Regrading claim 1, Raman teaches a non-transitory machine-readable medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, the operations comprising:
receiving input data relating to software applications in a predetermined format (Col. 2 lines 17-32: Describes a non-transitory CRM with processors to receive a log file that includes log records generated form a code base.; Col. 7 lines 43-67: Describes that the received log file and log records constitute input data relating to a software application where the data and artifact repository stores testing data for the application or system being tested, including requirements, production data, test cases, test results, application logs, trace data, and code quality analysis data.; Col. 11 lines 55-67 cont. Col. 12 lines 1-27; Describes that the software test scenarios may be written in a behavior driven development style using a formatted language, such as Gherkin, and may be included in a feature file used to capture requirements and define test scenarios. );
preprocessing the input data to extract a set of features (Col. 4, lines 60-67 cont. Col. 5, lines 1-22: Describes requirement documentation, including business requirements, functional requirements, use cases, and user stories, captures information relating to business processes supported by the software, intended actions performed through the software, managed data, rule sets, and non-functional attributes. It uses the documentation to extract key processes and events, feature requirements, non-functional testing, and other information through techniques including named entity recognition, topic segmentation, [part-of-speech tagging, terminology extraction, relationship extraction, semantic clustering, and semantic graphs.; Col. 11, lines 46-67 cont. Col. 12, lines 1-4: Describes a terminology extractor parses requirements documents and extracts key terminologies/extracted terms pertaining to processes, operations, actions, and flow and control of data.);
producing a label for each input data to generate a set of labels (Col. 8 lines 49-67 cont. Col 9, lines 1-12: Describes that the terminology extractor module extracts terminologies form a requirements document, such as terminologies pertaining to business processes, operations, and/or data and control, and feeds the extracted information into classifier modules. Each of the classifiers classifies the received terms into categories such as business processes, operations and actions, or data sets, using techniques such as topic segmentation.; Col. 10, lines 3-17: Describes that test scenario element classifier module 270 classifies extracted terms 262 into process terms 272, operations terms 274, and data set terms 276, corresponding to producing labels/classifications for the received input data to generate a set of labels.);
Although Raman teaches producing a label for each input data to generate a set of labels it does not teach constructing a training data set based on an extracted set of features and the set of labels.
Nogueira constructing a training data set based on an extracted set of features and the set of labels ([0020-0025]: Describes a system in which a training set and a predefined set of class labels are input to a learning algorithm to train a model, and input text is provided to the trained model To generate a predicted class label of the input text the input features include word embeddings, where words are mapped to multidimensional vectors, in that the CRCNN Transforms words in a text string into real valued feature vectors.; [0041]: Describes receiving a set of training data including, for each training around, training text, a correct class label that correctly classifies the training text, and incorrect class label that incorrectly classifies the training text, or the class labels are selected from the class labels identifying the predefined set of classes.);
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the machine learning software testing system of Raman with the supervised text classification of Nogueira. Doing so would have enabled the system to accelerate testing automation, improving decision making, and improve the productivity and quality of test case identification.
training a machine learning model with the training data set ([0022]: Describes that a training set including training data is input to a learning algorithm along with a predefined set of class labels to train a model.; [0042]: The CR-CNN is trained using the contents of the set of training data, and that the training includes learning parameters of the convolutional layer and parameters of class distributed vector representations (DVRs) for each of the predefined set of classes.);
generating a classifier based on the trained machine learning model ([0022]: Describes that the learning algorithm trains model 110, in that model 110 includes the trained CR-CNN including the class embedding matrix, which is compared to a drive of input text to generate a predicted class label of the text.; [0046]: Describes that the CR-CNN receives input text, generates a DVR based on the input text, compares the DVR to each of the class DVRs to generate a score for each class, and Outputs a predicted class label of the text. Thus, the trained CR-CNN model corresponds to a classifier generated based on the train machine learning model.);
providing, to the classifier, new input data relating to the software applications ([0022]: Describes that input text 108 is input to the model 110 to generate a predicted class label of the input text.; [0046]: Describes that input text 108 is received By the CR-CNN model 110, and that the CR-CNN generates a DVR based on the input text and outputs a predicted class label of the text.; Raman Col. 7, lines 43 cont. Col 8. lines 1-13: Describes that the input data relates to software applications, including requirements documentation, use cases, user stories, test cases, application logs, production data, test results, and other testing data for an application or system based being tested.);
Raman in view of Nogueira teaches generating, by the classifier, a predicted label identifying automated tests in response to the new input data (Raman Col. 15 line 59 cont. Col. 16, lines 1-40: Describes that failure prediction module 640 predicts which functionalities in the tested application or system are likely to fail, and that graphical representation generator module 650 generates graphical representations depicting relative test priories of functionalities and associated test cases based on the prediction. Dynamic test case selector and sequencer engine 126 selects a next set of test cases to be executed, and that test case selector module 710 selects test cases to be executed form the test priority graphical representations. Which teaches that the predicted label identifies automated tests.; Nogueira [0022]: Describes that input text 108 is input to model 110 to generate a predicted class label of the input text.; [0046]:Describes that the CR-CNN/model receives input text, generates a DVR based on the input text, compares the DVR to class DVRs to generate A score for each class, and outputs the predicted class label of the text. Which teaches generating, by the classifier, a predicted label in response to the new input data.);
Although Raman in view of Nogueira teaches, generating, by the classifier, a predicted label identifying automated tests in response to the new input data. They do not teach providing the automated tests to a continuous integration and continuous deployment (CI/CD) pipeline.
Sharma teaches, providing the automated tests to a continuous integration and continuous deployment (CI/CD) pipeline (Col. 3, lines 42-67: Describes a software test automation framework including automated test cases, automated test suites, an automation server, and test execution results. The framework resides in a software configuration management repository such as GitHub and can be run using a test automation server such as Jenkins, and that the automation server can execute test suites to run automated rests or build software. Corresponding to a continuous integration and continuous deployment pipeline being provided automated tests.).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of Raman in view of Nogueira with the automated pipeline of Sharma. Doing so would have allowed the system to reduce manual intervention, and enhance automated software testing.
Regarding claim 2 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Raman teaches the generating the classifier further comprises generating a plurality of classifiers (Col. 8 lines 49 cont. Col. 9, lines 1-14: Describes that the terminology extractor module 210 extracts terminologies form a requirements document, and feeds the extracted information into three classifier modules. Each of the three classifiers classifies the received terms into categories and that other classifier modules may also be employed.) and the generating the predicted label further comprises identifying, by the plurality of classifiers, a plurality of attributes of the automated tests, responsive to the new input data relating to the software applications (Raman Col. 10 lines 3-17, and Col. 10, lines 31 cont. Col. 11, lines 1-20 :Describes a plurality of classifier modules which classify received terms into categories. They classify extracted terms int process terms, operation terms,, and data set terms. The resulting test scenario/feature files are used to generate automated testing scripts for automation tools.; Nogueira as described above [0022], and [0046]: Describes that input text is input to the model to generate a predicted class label of the input text, and that the CR-CNN/model receives input test, generates a DCR based on the input test, compares the DVR to clad DVRs and outputs the predicted class label of the text.).
Regarding claim 3 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 2.
Raman teaches further comprise, among the plurality of attributes of the automated tests:
identifying, with a first classifier, a first attribute of the automated tests that corresponds to a testsuite to run;
identifying, with a second classifier, a second attribute that corresponds to a service; and
identifying, with a third classifier, a third attribute that corresponds to an action (Col. 8 lines 49 cont. Col. 9, lines 1-14, and Col. 10, lines 31-67: Describes identifying, with different classifiers, different attributes of automated tests. Its business process classifier 220, operations 222, and data set classifier 224, which classify extracted terms into categories including business processes, operations/actions, and data sets. It selects test cases form a plurality of test cases and transmitting the selected test cases to a test execution engine. Thus, it identifies different testing attributes, including a test/testsuite attribute, a process/service attribute, and an operation/action attribute.).
Regarding claim 4 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 3.
wherein the operations further comprise performing a postprocessing that concatenates the first attribute, the second attribute and the third attribute into a single prediction (Col. 10, lines 31 cont. Col. 11, lines 1-20: Describes that test scenario map builder 290 uses generated semantic graphs and process maps to generate test scenario maps for the respective requirements documentation, where the semantic graphs and process maps include processes and functionality that may be testes, valid and invalid operations for each functionality, expected outputs, and relationships between the various process. The requirements statements may be parsed to extract Intent, Objects, Actions, expected results, and test data elements, and that combinations of these extracted elements are used by the test scenario map builder to build complete statements. ).
Regarding claim 8 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Nogueira teaches, wherein the training the machine learning model with the training data set further comprises training the machine learning model using supervised machine learning by using the extracted set of features as an input and the set of labels associated with the extracted set of features as an output ([0007], and [0020-0022]: Describes that input features for the CNN include word embeddings and that the CR-CNN uses word embedding features to perform text classification. A training set including training data is input to a learning algorithm along with a predefined set of class labels to train a model, where the learning algorithm learns a class embedding matric based on word embedding features of the training data.; [0041-0042]: Describes that the training data includes training text, a correct class label that correctly classifies the training text, and an incorrect class label that incorrectly classifies the training text, and that the CR-CNN is trained using the contents of the set of training data. Corresponding to a supervised machine learning using extracted features as input and associated labels as output.).
Regarding claim 9 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Sharma teaches wherein the operations further comprise generating a notification that the automated tests executed in the CI/CD pipeline have passed or failed (Col. 3, lines 42 cont. Col. 4, lines 1-18: Describes automated test suites which are run through an automation server, such as Jenkins, to run automated test and provide desired test results. The test automation results are obtained from a Jenkins server location, parsed to identify failures or exceptions for respective tests, and recorded for further processing.; Col. 5, lines 54: Describes that an automated notification system notifies stakeholders via email or instant messaging about the status of the bug ticket resulting from the failure/exception.).
Regarding claim 10 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Raman teaches wherein the receiving the input data relating to the software applications in the predetermined format further comprises receiving the input data in a text form, an image form, an audio form, a video form, or a combination thereof (Col. 4, lines 60 cont. Col. 5, lines 1-22: Describes receiving and processing requirements documentation including business requirements, functional requirements, use cases, user stories, and other written information relating to business processes supported by the software, intended actions performed through the software, managed data, rule sets, and non-functional attributes. It uses natural language processing o such requirements documentation to extract key processes, feature requirements, and related information.).
Regrading claims 12, which recites substantially the same limitations as claim 8 but further recite a device, comprising: a processing system including a processor; and a memory that stores executable instructions (Raman Col. 1, lines 35-67: Describes how their processes can be implemented on computing devices.) to perform the steps of claim 8, respectively, and are rejected for the same reasons as described in this office action.
Regrading claims 16, 17, and 19 which recites substantially the same limitations as claims 2, 3, and 8 but further recite a method (Raman Col. 2, lines 4-16: Describes how the steps are a computer implemented method) to perform the steps of claims 2, 3, and 8, respectively, and are rejected for the same reasons as described in this office action.
Claim(s) 5, 7, 11, 14, 15, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Raman et al. (US 10073763 B1, referred to as Raman), in view of Nogueira dos Santos et al. (US 20170308790 A1, referred to as Nogueira), in view of Sharma et al. (US 10572374 B2, referred to as Sharma), in view of Zare et al. (US 11461646 B2, referred to as Zare).
Regarding claim 5 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Although they teach the non-transitory machine readable medium of claim 1 they do not teach testing the classifier with another set of training data; and upon testing of the classifier, determining that an accuracy score of the classifier exceeds a predetermined threshold.
Zare teaches testing the classifier with another set of training data; and
upon testing of the classifier, determining that an accuracy score of the classifier exceeds a predetermined threshold (Col. 5, lines 1-39: Describes creating training sets and validation sets from an original training dataset, where each trainings et is associated with a validation set.; Col. 6 lines 1-47: Describes training machine learning models using the training and validation datasets and evaluating model accuracy using the validation set associated with the training set.; Col. 7, lines 15-29: Describes determining whether the machine learning models satisfy an accuracy criterion, where the accuracy criterion may be a threshold value, and using models having accuracy metric values that satisfy the accuracy criterion, such as exceeding a predetermined threshold.).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of Raman in view of Nogueira, in view of Sharma with the classifier validation and accuracy threshold of Zare. Doing so would have allowed the system to improve the reliability of the automated test selection process.
Regarding claim 7 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Although they teach the non-transitory machine readable medium of claim 1 they do not teach constructing a second training data set based on different data from the input data; and testing the classifier with the second training data set by comparing a predicted automated test by the classifier with an actual test in response to the second training data set.
Zare teaches, constructing a second training data set based on different data from the input data; and
testing the classifier with the second training data set by comparing a predicted automated test by the classifier with an actual test in response to the second training data set (Col. 5, lines 1-59: Describes creating a plurality of training sets and validation sets form an original training dataset, where data points are assigned to a training set or a validation set, and each training set is associated with a validation set.; Col. 6, lines 26-47: Describes that, after a model is created using a training set, the model accuracy is evaluated using the validation set associated with the training set, and that the accuracy metric reflects how accurate each model predicts the validation data provided to it. ).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of Raman in view of Nogueira, in view of Sharma with the classifier validation of Zare. Doing so would have allowed the system to improve the training process by verifying the classifier with new data.
Regrading claim 11, which recites substantially the same limitations as claims 1, 7 and 8 but further recite a device, comprising: a processing system including a processor; and a memory that stores executable instructions (Raman Col. 1, lines 35-67: Describes how their processes can be implemented on computing devices.) to perform the steps of claim 1, 7, and 8 respectively, and are rejected for the same reasons as described in this office action.
Regrading claim 14, which recites substantially the same limitations as claim 7 but further recite a device, comprising: a processing system including a processor; and a memory that stores executable instructions (Raman Col. 1, lines 35-67: Describes how their processes can be implemented on computing devices.) to perform the steps of claim 7, respectively, and are rejected for the same reasons as described in this office action.
Regrading claim 15 which recites substantially the same limitations as claims 1, 7 and 8 but further recite a method (Raman Col. 2, lines 4-16: Describes how the steps are a computer implemented method) to perform the steps of claims 1, 7 and 8, respectively, and are rejected for the same reasons as described in this office action.
Regrading claim 18 which recites substantially the same limitations as claims 1-4 and 8 but further recite a method (Raman Col. 2, lines 4-16: Describes how the steps are a computer implemented method) to perform the steps of claims 1-4, respectively, and are rejected for the same reasons as described in this office action.
Claim(s) 6, 13, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Raman et al. (US 10073763 B1, referred to as Raman), in view of Nogueira dos Santos et al. (US 20170308790 A1, referred to as Nogueira), in view of Sharma et al. (US 10572374 B2, referred to as Sharma), in view of Grewal et al. (US 20220210188 A1, referred to as Grewal).
Regarding claim 6 Raman in view of Nogueira, in view of Sharma teaches the non-transitory machine-readable medium of claim 1.
Raman teaches wherein the input data includes text data (Col. 4, lines 60 cont. Col. 5, lines 1-22 : Describes that requirements documentation including business requirements, functional requirements, use cases, user stores, and other written information relating to business processes supported by the software, intended actions performed through the software, managed data, rule sets, and non-functional attributes. It uses natural language processing on the requirements documentation. );
Although they teach the non-transitory machine readable medium of claim 1 they do not teach the preprocessing the input data further comprises generating reduced text data by: tokenizing the input data into each word; removing one or more stopwords from the extracted set of features; and lemmatizing the input data.
Grewal teaches the preprocessing the input data further comprises generating reduced text data by:
tokenizing the input data into each word;
removing one or more stopwords from the extracted set of features; and
lemmatizing the input data ([0050]: Describes generating reduced text data by preprocessing body text, including removing stop words and performing lemmatization, and then performing tokenization to convert sentences in the body text to individual words).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the system of Raman in view of Nogueira, in view of Sharma with the test preprocessing of Grewal. Doing so would have allowed the system to reduce irrelevant words, and sort input data to better enhance the classifier processing and speed.
Regrading claim 13, which recites substantially the same limitations as claim 6 but further recite a device, comprising: a processing system including a processor; and a memory that stores executable instructions (Raman Col. 1, lines 35-67: Describes how their processes can be implemented on computing devices.) to perform the steps of claim 6, respectively, and are rejected for the same reasons as described in this office action.
Regrading claim 20 which recites substantially the same limitations as claim 6 but further recite a method (Raman Col. 2, lines 4-16: Describes how the steps are a computer implemented method) to perform the steps of claim 6, respectively, and are rejected for the same reasons as described in this office action.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892 for additional art including.
US 11423330 B2: comparing predicted outputs to correct outputs
US 11221942 B2 software test automation
US 10515002 B2: generating test cases
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONALD T RODEN whose telephone number is (571)272-6441. The examiner can normally be reached Mon-Thur 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T.R./Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights