DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 14 is objected to because of the following informalities: Claim 14 claims, “The computer program product of claim 1”. However, the examiner believes claim 14 should be dependent on claim 11 not claim 1, “The computer program product of claim 11”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly
claiming the subject matter which the applicant regards as his invention.
Claim 1 recites the limitation "the one or more first inputs" in lines 4-5. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 claims, “receiving, by the test manager engine via a prompt, one or more second inputs comprising a user text description of the product or the feature of the product in the prompt that alter the textual description to a test description of the product or the feature of the product; and”. This step is currently unclear to the examiner. It is unclear to the examiner if “the textual description to a test description” in line 8 is the same is a the same is “a textual description” in lines 3-4. The examiner believes the claim is trying to claim that the user text description is altering the textual description of lines 3-4 that is used to generate the test description. However, as claimed the limitation is unclear.
Claims 4 and 14, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. For instance, claim 4 claims, “…wherein the model comprises…”. Claim 14 has similar language. The examiner believes, this limitation is referring to the “artificial intelligence model” in claim 1. However, currently as claimed it is unclear.
Claim 9, is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 9 claims, “…wherein the model comprises…”. The examiner believes, this limitation is referring to the “artificial intelligence model” in claim 1. However, currently as claimed it is unclear.
Claim 10, is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 10 claims, “…provide the textual description within the prompt of the figma file.”. This limitation is currently unclear. Claim 1 claims, “receiving…via a prompt, one or more second inputs comprising a user text description of the produce or the feature of the product in the prompt…”. It is unclear to the examiner if the “prompt of the figma file” is the same as the “prompt” of claim 1. It is currently unclear what the “prompt of the figma file” is.
Claims 2-3 and 5-8 are rejected for being dependent on rejected claim 1.
Claims 11-20 contain similar limitations to claim 1-9 and are therefore rejected for the same reasons.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claim 1 recites, “generating…a textual description of a produce or a feature of the produce from the one or more first inputs…” and “automatically generating…one or more test cases”. The limitations of “generating” and “generating” as drafted are functions that, under their broadest reasonable interpretation, recite the abstract idea of a mental process. The limitations encompass a human mind carrying out the function through observation, evaluation, judgment and /or opinion, or even with the aid of pen and paper. Thus, this limitation recites and falls within the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The claim recites the following additional elements “receiving…one or more second inputs…”. The additional element of “receiving” is an insignificant pre solution activity. See MPEP 2106.05(g). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claim is therefore directed to the judicial exception.
Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Regarding the limitations of ““receiving…one or more second inputs…”, the courts have identified mere data gathering as well-understood, routine and conventional activity. Se MPEP 2106.05(d) and MPEP 2106.05(f). The recitation of generic computer instruction and computer components to apply the judicial exception, and the well-understood, routine, conventional activities do not amount to significantly more, thus, cannot provide an inventive concept. Accordingly, claim 1 is not patent eligible under 35 USC 101.
Claim 2, claims “receiving one or more first inputs…”. The courts have identified mere data gathering as a well-understood, routine and conventional activity. See MPEP 2106.05(d) and MPEP 2106.05(f). The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claims 3, further defines the first inputs. The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 4, claims “processing the one or more inputs…”. The step of “processing” is an additional limitation of the abstract idea “Mental Process”. Nothing in the claimed limitation prevents it from being performed in the mind. Further the additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 5, claims “…delegate operations…”. The step of “delegate” is an additional limitation of the abstract idea “Mental Process”. Nothing in the claimed limitation prevents it from being performed in the mind. Further the additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 6, further defines the artificial intelligence model. Nothing in the claimed limitation prevents it from being performed in the mind. Further the additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 7, claims “…providing the textual description of the produce…in the prompt”. This limitation is identified as well-understood, routine, conventional activity (2106.05(d) and 2106.05(g)). The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 8, claims “… the textual description being displayed in the prompt.”. Displaying is identified as well-understood, routine, conventional activity (2106.05(d) and 2106.05(g)). The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 9, further defines the model. The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claim 10, further defines the generation step of claim 1. The additional elements are neither a practical application under prong 2, nor an inventive concept under step 2B.
Claims 11-20 contain similar limitations to claim 1-9 and are therefore rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Mani et al. (US 10,698,803 B1) further in view of OpenAI, (GPT-4 Technical Report”, 2023).
As per claim 1, Mani et al. teaches the invention as claimed including, “A method executed by a test manager engine implemented as a computer program, the method comprising:
generating, by the test manager engine utilizing an artificial intelligence model, a textual description of a product or a feature of the product from the one or more first inputs;”
Visual inputs from a specification document may be transmitted to or otherwise obtained by test script generating tool. The test script generating tool uses visual inputs to generate a test script that may be used to test computer programs developed using the specification document. The test script generating tool uses a conversion engine to convert each of the visual inputs into textual objects, which are text based objects that encoded the functional elements represented by visual inputs. The test script generation tool uses the textual objects with an identifier tool to identify the properties of textual objects and whether they match with known computer code elements. Then the test script generating tool uses the script engine to generate test scripts (column 4, lines 43 – column 5, lines 1-3).
Conversion engine of test script generating tool uses the plurality of visual inputs to generate corresponding textual objects. Converting visual inputs to textual objects may include identifying pictorial elements by classification and rendering those classifications into a respective one of textual objects, such as a string or other text-based data structure. Conversion engine may convert visual inputs into textual objects that represent the requirements of specification document. Textual object may be more easily handled by test script generating tool (column 7, lines 22-48). Also see column 14, lines 21-34 and figure 3.
The test script generating tool incorporates one or more machine learning techniques to identify elements from the visual inputs of the computer program specification document. In certain embodiments, the test script generating tool uses Bayesian analysis to determine the type and characteristics of an element in the visual input. Further in some embodiments, the test script generating tool uses keyword-based sentimental classification to identify an element in the visual input based on the text identified from the visual input (column 2, lines 19-35). Conversion engine may handle a variety of types of visual inputs and generate different types of textual objects. Visual inputs are mapped to textual objects (column 7, lines 49-54). Textual object may be more easily handled by test script generating tool (column 7, lines 22-48). However, Mani et al. does not explicitly appear to teach, “generating, by the test manager engine utilizing an artificial intelligence model, a textual description of a product or a feature of the product from the one or more first inputs;”
OpenAI, teaches a GPT-4 model that is able to process image and text inputs and produce text outputs (1 Introduction).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mani et al. with OpenAI. Mani et al. teaches a conversion engine may generate textual objects based on visual data. Textual objects may include a string of test that describe or represent visual input. Textual objects encode the functional information from the specification document displayed in visual input in a text form (column 14, lines 21-34). Visual inputs are mapped to textual objects (column 7, lines 49-54). Mani et al. further teaches its test script generating tool incorporates one or more machine learning techniques to identify elements from the visual inputs of the computer program specification document (column 2, lines 19-35) and that the textual object may be more easily handled by the test script generating tool (column 7, lines 2-48). OpenAI, teaches a GPT-4 model that is able to processing image and text inputs and produce text outputs (1 Introduction). Therefore, they both teach converting an image into a textual output. Substituting the GPT-4 model for the conversion engine of Mani et al. would achieve similar results and therefore encode the visual input into text objects.
“receiving, by the test manager engine via a prompt, one or more second inputs comprising a user text description of the product or the feature of the product in the prompt that alter the textual description to a test description of the product or the feature of the product; and”
Mani et al. teaches, the system allows a user to interact with one or more components to improve and/or provide flexibility to generation of test scripts. Test script generating tool prompts the user with a user interface for an identification of a respective one of textual objects. If the identifier tool cannot determine a match for the respective textual object, it may request user input to specify any missing attributes or identifying a matching element (column 11, lines 24-60).
“automatically generating, by the test manager engine, one or more test cases for the product or the feature of the product from the test description.”
Mani et al. teaches the system is configured to take visual inputs from a specification document and generate one or more test scripts using a test script generation tool and use the generated test scripts with an automation framework to test computer program code (column 4,lines 20-26). Visual inputs from specification document may be transmitted to or otherwise obtained by test script generating tool. The test script generating tool uses visual inputs to generate a test script that may be used to test computer programs developed using the specification document. The test script generation tool uses the textual objects with an identifier tool to identify the properties of textual objects and whether they match with known computer code elements. Then the test script generating tool uses the script engine to generate test scripts (column 4, lines 43 – column 5, lines 1-3). Also see figure 3.
As per claim 2, Mani et al. further teaches, “The method of claim 1, wherein the method comprises receiving the one or more first inputs describing the product or the feature of the product.”
Visual inputs from specification document may be transmitted to or otherwise obtained by test script generating tool. The test script generating tool uses visual inputs to generate a test script that may be used to test computer programs developed using the specification document. The test script generating tool uses a conversion engine to convert each of the visual inputs into textual objects, which are test based objects that encoded the functional elements represented by visual inputs. The test script generation tool uses the textual objects with an identifier tool to identify the properties of textual objects and whether they match with known computer code elements. Then the test script generating tool uses the script engine to generate test scripts (column 4, lines 43 – column 5, lines 1-3). Conversion engine of test script generating tool uses the plurality of visual inputs to generate corresponding textual objects. Converting visual inputs to textual objects may include identifying pictorial elements by classification and rendering those classifications into a respective one of textual objects, such as a string or other text-based data structure. Conversion engine may convert visual inputs into textual objects that represent the requirements of specification document. Textual object may be more easily handled by test script generating tool (column 7, lines 22-48). Conversion engine may generate textual objects based on visual data. Textual objects may include a string of test that describe or represent visual input. Textual objects encode the functional information from the specification document displayed in visual input in a text form (column 14, lines 21-34). Visual input may be visual input obtained from specification document that includes user design interfaces. For example, visual input may include elements that correspond to pictorial elements (e.g., arrows, wireframe buttons, decision elements, etc) and text elements (e.g., names, descriptors of elements, text describing functionality, yes/no forks, etc.) (column 14, lines 1-10). Also see figure 3.
As per claim 3, Mani et al. further teaches, “The method of claim 1, wherein the one or more first inputs comprise at least one of an image file, a wireframe drawing file, and a visual description file for the product or the feature of the product.”
Conversion engine may handle a variety of types of visual inputs and generate different types of textual objects. Visual inputs are mapped to textual objects (column 7, lines 49-54). Visual input may be visual input obtained from specification document that includes user design interfaces. For example, visual input may include elements that correspond to pictorial elements (e.g., arrows, wireframe buttons, decision elements, etc) and text elements (e.g., names, descriptors of elements, text describing functionality, yes/no forks, etc.) (column 14, lines 1-10).
As per claim 4, Mani et al. further teaches, “The method of claim 1, wherein the method comprises processing the one or more first inputs describing the product or the feature of the product to generate one or more text outputs, wherein the model comprises at least one of an optical character recognition model, a document understanding model, and a computer vision model.”
Conversion engine of test script generating tool uses the plurality of visual inputs to generate corresponding textual objects. Converting visual inputs to textual objects may include identifying pictorial elements by classification and rendering those classifications into a respective one of textual objects, such as a string or other text-based data structure. Conversion engine may convert visual inputs into textual objects that represent the requirements of specification document (column 7, lines 22-48).
As per claim 6, OpenAI further teaches, “The method of claim 1, wherein the artificial intelligence model comprises generative pretrained transform.”
OpenAI, teaches a GPT-4 (generative pretrained transformer) model that is able to process image and text inputs and produce text outputs (1 Introduction).
As per claim 7, Mani et al. further teaches, “The method of claim 1, wherein the method comprises providing the textual description of the product or the feature of the product in the prompt.”
Mani et al. teaches, the system allows a user to interact with one or more components to improve and/or provide flexibility to generation of test scripts. Test script generating tool prompts the user with a user interface for an identification of a respective one of textual objects. If the identifier tool cannot determine a match for the respective textual object, it may request user input to specify any missing attributes or identifying a matching element. Script engine generated the test script (column 11, lines 24-60). User may customize the test script e.g., by selecting certain identified functions or elements to test or not test. Based on this user input, the test script generating tool may generate a test script that is tailored for the testing desired for the computer program. Furthermore, user input may be used by the test script generating tool to enhance its machine learning engines. For example, the user may add additional contextual information though with which the test script generating tool may identify the elements based on the visual input from the computer code specifications document (column 2, lines 45-59).
As per claim 8, Mani et al. further teaches, “The method of claim 1, wherein the one or more second inputs are provided into the prompt to directly interact with the textual description being displayed in the prompt.”
Mani et al. teaches, the system allows a user to interact with one or more components to improve and/or provide flexibility to generation of test scripts. Test script generating tool prompts the user with a user interface for an identification of a respective one of textual objects. If the identifier tool cannot determine a match for the respective textual object, it may request user input to specify any missing attributes or identifying a matching element. Script engine generated the test script (column 11, lines 24-60). User may customize the test script e.g., by selecting certain identified functions or elements to test or not test. Based on this user input, the test script generating tool may generate a test script that is tailored for the testing desired for the computer program. Furthermore, user input may be used by the test script generating tool to enhance its machine learning engines. For example, the user may add additional contextual information though with which the test script generating tool may identify the elements based on the visual input from the computer code specifications document (column 2, lines 45-59).
As per claim 9, Mani et al. further teaches, “The method of claim 1, wherein the model comprises an optical character recognition model that processes a figma file as the one or more first inputs to generate an optical character recognition result.”
Visual inputs may include portions of a flow diagram describing how to implement functional blocks of the computer program or certain style or user design specifications. As a result, visual inputs may represent the functional requirements of the computer program to be developed (column 4, lines 30-43). Conversion engine may handle a variety of types of visual inputs and generate different types of textual objects. Visual inputs are mapped to textual objects (column 7, lines 49-54). Visual input may be visual input obtained from specification document that includes user design interfaces. For example, visual input may include elements that correspond to pictorial elements (e.g., arrows, wireframe buttons, decision elements, etc) and text elements (e.g., names, descriptors of elements, text describing functionality, yes/no forks, etc.) (column 14, lines 1-10).
Conversion engine of test script generating tool uses the plurality of visual inputs to generate corresponding textual objects. Converting visual inputs to textual objects may include identifying pictorial elements by classification (OCR) and rendering those classifications into a respective one of textual objects, such as a string or other text-based data structure. Conversion engine may convert visual inputs into textual objects that represent the requirements of specification document. Textual object may be more easily handled by test script generating tool (column 7, lines 22-48).
It would have been obvious to one of ordinary skill in the art before the effective filing ate for one for the first inputs to be a figma file. Mani et al. teaches a conversion engine may handle a variety of types of visual inputs and generate different types of textual objects. Visual inputs are mapped to textual objects (column 7, lines 49-54). Visual input may be visual inputs obtained from specification document that includes user design interfaces. For example, visual input may include elements that correspond to pictorial elements (e.g., arrows, wireframe buttons, decision elements, etc) and text elements (e.g., names, descriptors of elements, text describing functionality, yes/no forks, etc.) (column 14, lines 1-10). Therefore, Maini et al. teaches the visual input can be of a variety of types including specification documents that include user design interfaces and wireframe buttons. A figma file is a know type of file that give a visual description of the design of an application. This may include a wireframe. Therefore, it would have been obvious for it to be the input. This is nothing more that a design choice and would have been obvious to try.
As per claim 10, Mani et al. and OpenAI further teach, “The method of claim 9, wherein the optical character result is provided to a generative pretrained transform of the artificial intelligence model to provide the textual description within the prompt of the figma file.”
Conversion engine of test script generating tool uses the plurality of visual inputs to generate corresponding textual objects. Converting visual inputs to textual objects may include identifying pictorial elements by classification (OCR) and rendering those classifications into a respective one of textual objects, such as a string or other text-based data structure. Conversion engine may convert visual inputs into textual objects that represent the requirements of specification document. Textual object may be more easily handled by test script generating tool (column 7, lines 22-48).
OpenAI, teaches a GPT-4 (generative pretrained transformer) model that is able to process image and text inputs and produce text outputs (1 Introduction).
As per claims 11-14 and 16-20, they contain similar limitations to claims 1-4 and 6-10 and are therefore rejected for the same reasons.
Claims 5, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Mani et al. (US 10,698,803 B1) and OpenAI, (GPT-4 Technical Report”, 2023) as applied to claims 1 and 11 above, and further in view of Doyle et al. (US 20250307436 A1).
As per claim 5, Mani et al. teaches a test script generation tool that perform the steps of generating a test script from textual objects. However, Mani and OpenAI do not explicitly appear to teach, “The method of claim 1, wherein the test manager engine comprises a master-client architecture that delegates operations of the artificial intelligence model to a client artificial intelligence agent.”
Doyle et al. teaches an AI agent that can fetch a required AI module from a knowledge module registry, prove the right to use it, execute the AI module on local data (abstract). Also see figure 2 and claim 8.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mani et al. and OpenAI with Doyle et al. All three teach the use of Artificial Intelligence models. Mani et al. teaches a test script generation tool that controls the use of a identifier tool 120 thats using a machine learning model (figure 1). OpenAI teaches the use of a generative pre-trained transformer (AI) that can transform an image into a text. Doyle et al. teaches an AI agent that is able to fetch a required AI module when needed, prove the rights to use it and execute the AI module on local data. This would allow Mani et al. call/request different AI modules according to the type of task it wants to perform such as a type of input such as image file, visual description file, wireframe drawing file or the like. It is nothing more than a design choice and would have been obvious to try.
As per claim 15, it contains similar limitations to claim 5 and is rejected for the same reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Duan et al. (“Generating Automatic Feedback on UI Mockups with Large Language Models”, arXiv, 2024). Duan et al. teaches a Figma plugin that take a UI design and a set of written heuristics and renders automatically generated feedback as constructive suggestions (Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK A GOORAY whose telephone number is (571)270-7805. The examiner can normally be reached Monday - Friday 10:00am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lewis Bullock can be reached at 571-272-3759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK A GOORAY/ Examiner, Art Unit 2199
/LEWIS A BULLOCK JR/ Supervisory Patent Examiner, Art Unit
Prosecution Insights