Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note (EN)
The prior art rejections below cite particular paragraphs, columns, and/or line numbers in the references for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1: Statutory class – apparatus.
Step 2A Prong One: Does the claim recite an abstract idea, law of nature or natural phenomenon?
Yes
“3) Mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III).” MPEP § 2106.04(a).
The claims are directed to an abstract idea of data processing and analysis. The claim recites:
validate the updated user interface based on a comparison between the screenshot and a reference image associated with the updated user interface.
The measuring, calculating and estimating limitations are limitations of mental processes of evaluation, and judgement. By way of example, one can mentally evaluate the quality metrics of the source code, calculate a complexity score, and calculate a trend approximation along with a lifetime estimate based on the previously calculated values.
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
No.
The additional elements are:
a processor;
a memory coupled to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the system to:
cause to make a change to a document object model (DOM) of a software application, the DOM of the software application being associated with a user interface of the software application;
save the change associated with the DOM;
render an updated user interface based on the changed DOM;
capture a screenshot of the updated user interface; and
The memory / processor and circuitry / components are generic computer components used as a tool. They provide nothing more than mere instructions to implement an abstract idea on a generic computer. The causing a change – data entry, saving – data output, rendering – data output, and capturing – data entry limitations are insignificant extra-solution activity that does not produce a practical application of the abstract idea or amount to significantly more. See MPEP 2106.05(d).
Step 2B: Does the claim recite additional elements that amount to significantly more than judicial exception?
No, as discussed with respect to Step 2A, the additional limitation are instructions to apply an exception on a generic computer and a general purpose computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer.
Additionally, the limitations do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B. Further, in regards to step 2B MPEP 2106.05 (d) - II. ELEMENTS THAT THE COURTS HAVE RECOGNIZED AS WELL-UNDERSTOOD, ROUTINE, CONVENTIONAL ACTIVITY IN PARTICULAR FIELDS
Because examiners should rely on what the courts have recognized, or those of ordinary skill in the art would recognize, as elements that describe well‐understood, routine activities, the following section provides examples of elements that have been recognized by the courts as well-understood, routine, conventional activity in particular fields. It should be noted, however, that many of these examples failed to satisfy other considerations (e.g., because they were recited at a high level of generality and thus were mere instructions to apply an exception, or were insignificant extra-solution activity). Thus, examiners should carefully analyze additional elements in a claim with respect to all relevant Step 2B considerations, including this consideration, before making a conclusion as to whether they amount to an inventive concept.
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
• i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
• ii. Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012) ("The computer required by some of Bancorp’s claims is employed only for its most basic function, the performance of repetitive calculations, and as such does not impose meaningful limits on the scope of those claims.");
• iii. Electronic recordkeeping, Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 225, 110 USPQ2d 1984 (2014) (creating and maintaining "shadow accounts"); Ultramercial, 772 F.3d at 716, 112 USPQ2d at 1755 (updating an activity log);
• iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.
The additional elements have been considered both individually and as an ordered combination in the significantly more consideration. This claim is ineligible.
Claim 2 recites identify a set of static features and a set of variable features of the updated user interface, which is a mental/mathematical process under Step 2A Prong One.
mask the set of variable features from the screenshot of the updated user interface, which is mere data entry under Step 2A Prong Two and 2B.
perform image matching techniques to compare the screenshot and the reference image, which is a mental/mathematical process under Step 2A Prong One.
Therefore, the claim is considered ineligible under 35 USC 101.
Claim 3 recites generate a test report for the changed DOM based on the comparison, the test report indicating a similarity value for the screenshot and the reference image, which is a mental/mathematical process under Step 2A Prong One.
Claim 4 recites wherein the reference image is stored in a database, which is mere data output under Step 2A Prong Two and 2B. Therefore, the claim is considered ineligible under 35 USC 101.
Claims 5-8 are medium claims reciting limitations similar to claims 1-4 respectively and are rejected under the same rationale.
Claims 9-12 are method claims reciting limitations similar to claims 1-4 respectively and are rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over Hey et al. (US20150347284A1) in view of Liu et al. (US20110258538A1)
Regarding Claim 1, Hey teaches A system comprising: a processor; a memory coupled to the processor, the memory having stored therein at least one of programs or instructions executable by the processor to configure the system to (Hey, Fig. 2, [0052-0054])
capture a screenshot of the updated user interface (Hey, [0021] "User interface 112 initiates automated test scenario 118. Test scenario 118 consists of a series of executable commands to access and update test page 114 and execute screenshot verification program 120. Test scenario 118 may contain information regarding scope and page objects. The scope is an area within a webpage that can be defined by a test scenario which identifies the outer bounds of page objects. Page objects are models of areas within a webpage application that interact with a user or test. In some embodiments, test scenario 118 identifies page objects and an associated scope to screenshot verification program 120 for data collection and comparison. In other embodiments, test scenario 118 identifies some page objects and an associated scope to screenshot verification program 120, creating a partial screenshot. A partial screenshot is a section or portion of a full screenshot, containing one or more page objects, as identified by test scenario 118. Screenshot verification program 120 excludes unidentified page objects from data collection and comparison. In some other embodiments, test scenario 118 identifies a continuously updating dynamic page object (e.g. time and date stamp) to screenshot verification program 120 to exclude from data collection and comparison. A dynamic page object has the capability to change value independent of user input based upon a webpage function. Depending upon which function, master mode function 200 or test mode function 300, is activated by user interface 112, results obtained from screenshot verification program 120 are stored within storage device 116 as master screenshot 122 or test screenshot 124, respectively. Upon completion, screenshot verification program 120 determines success or failure of a test scenario, such as test scenario 118, based on whether a webpage, application, or feature is working as designed.")
validate the updated user interface based on a comparison between the screenshot and a reference image associated with the updated user interface (Hey, [0036-0037] "In step 302, test mode function 300 receives updated conditions upon continuation of a test scenario. In one embodiment, test mode function 300 receives a command to execute at the time a request initiates from user interface 112 on computing device 110 after user interface 112 establishes updated conditions. In another embodiment, test mode function 300 receives a command to execute, including updated conditions at the time a request initiates from within an automated process previously initiated on computing device 110. In the current embodiment of the invention presented, computing device 110, via input through user interface 112, accesses test scenario 118 and automates the process for test mode function 300, thus expediting the webpage testing. In one embodiment, test scenario 118 sets the updated conditions for test mode function 300. In another embodiment, updated conditions could be executed through user interface 112 directly. In step 304, test mode function 300 accesses a master screenshot which includes page objects and corresponding coordinate information from storage device 116. Test mode function 300 accesses master screenshot 122 in storage device 116. Master screenshot 122 identifies and defines page objects and coordinate information to test mode function 300 for reacquisition." Please see Fig. 3 and corresponding [0038-0046] for the validation decision tree/logic and subsequent report)
However, Hey doesn’t appear to explicitly disclose:
cause to make a change to a document object model (DOM) of a software application, the DOM of the software application being associated with a user interface of the software application
save the change associated with the DOM
render an updated user interface based on the changed DOM
Liu teaches cause to make a change to a document object model (DOM) of a software application, the DOM of the software application being associated with a user interface of the software application (Liu, [0027-0030] "Documents in DOM-based editors (editors) 22 may consist of text nodes and elements which modify the representation of text nodes. As shown in FIG. 4, there are various possible types of edit steps 2 that can change a document, including a replacement edit step 20 and an element manipulation edit steps 6. Replacement edit steps 20 may consist of contiguous insertions, deletions, and/or replacements entered without an excessive pause or changing focus. For example, a user 446 may select a range “old, unnecessary text”, type “new stuff”, and then pause or click elsewhere in a document 480. When a user 446 types into an editor 22, the editor 22 collects a logical set of keystrokes into an edit step 2 which can later be undone or redone as a unit…In accordance with various aspects of the invention, following an edit step 2, the editor 22 updates the display, persists the edit step 2 to a persistence engine 438 and/or collaborators, appends the edit step 2 to an undo/redo queue 508, and performs any specialty manipulations supported by that editor 22 (e.g., Microsoft Word's “Smart Text” feature recognizes phone numbers, addresses, and Outlook contacts).")
save the change associated with the DOM (Liu, [0030 and 0033] "In accordance with various aspects of the invention, following an edit step 2, the editor 22 updates the display, persists the edit step 2 to a persistence engine 438 and/or collaborators, appends the edit step 2 to an undo/redo queue 508, and performs any specialty manipulations supported by that editor 22 (e.g., Microsoft Word's “Smart Text” feature recognizes phone numbers, addresses, and Outlook contacts). … As previously indicated, an embodiment of the present invention provides a DOM-based editor 22 with a edit capture extension 431 configured to capture edit steps 2 (i.e., changes to an editor DOM 26) with help from a store DOM 24 that is kept in sync with editor DOM 26. An embodiment provides that edit steps 2 are encoded as DOM coordinates 146, as discussed further below in conjunction with FIG. 5. An embodiment provides means for a persistence engine 438 to more efficiently handle the resulting edit steps 2. An embodiment provides means to keep empty DOM elements navigable while editing. An embodiment provides support for mutation events 434. An embodiment demonstrates how edit steps 2 can be used in lieu of the native undo/redo functionality of the editor 22. An embodiment provides means to group related changes for purposes of undo/redo. An embodiment provides means to navigate a tree of revisions by remembering the active branch.")
render an updated user interface based on the changed DOM (Liu, [0030] "In accordance with various aspects of the invention, following an edit step 2, the editor 22 updates the display, persists the edit step 2 to a persistence engine 438 and/or collaborators, appends the edit step 2 to an undo/redo queue 508, and performs any specialty manipulations supported by that editor 22 (e.g., Microsoft Word's “Smart Text” feature recognizes phone numbers, addresses, and Outlook contacts).")
Hey and Liu are analogous art because they are from the same field of endeavor in UI modification and validation. Liu teaches direct modification of the DOM to modify the UI elements. Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art, to combine Hey and Lui to utilize Lui’s known technique, of modifying the DOM to modify the UI, into the environment and processes of Hey which teaches tracking and validating changes to the UI in order to facilitate the UI modification and update process. Note MPEP 2143- (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 2, Hey in view of Liu teaches the system of claim 1. Hey teaches wherein, to perform the comparison between the screenshot of the updated user interface and the reference image, the processor is further configured to:
identify a set of static features and a set of variable features of the updated user interface (Hey, [0013] "Elements contained within a webpage can be static elements (e.g., unchanging, string constant) or dynamically generated elements (i.e., elements containing values that vary within each instance of the webpage). Due to the complex nature of a webpage and the verification testing needing to be performed, as recognized by the embodiments of this invention, tests that are currently time consuming and inflexible are addressed, thus increasing the reliability, independence, and flexibility of a webpage." and [0021])
mask the set of variable features from the screenshot of the updated user interface; (Hey, [0021] "User interface 112 initiates automated test scenario 118. Test scenario 118 consists of a series of executable commands to access and update test page 114 and execute screenshot verification program 120. Test scenario 118 may contain information regarding scope and page objects. The scope is an area within a webpage that can be defined by a test scenario which identifies the outer bounds of page objects. Page objects are models of areas within a webpage application that interact with a user or test. In some embodiments, test scenario 118 identifies page objects and an associated scope to screenshot verification program 120 for data collection and comparison. In other embodiments, test scenario 118 identifies some page objects and an associated scope to screenshot verification program 120, creating a partial screenshot. A partial screenshot is a section or portion of a full screenshot, containing one or more page objects, as identified by test scenario 118. Screenshot verification program 120 excludes unidentified page objects from data collection and comparison. In some other embodiments, test scenario 118 identifies a continuously updating dynamic page object (e.g. time and date stamp) to screenshot verification program 120 to exclude from data collection and comparison. A dynamic page object has the capability to change value independent of user input based upon a webpage function. Depending upon which function, master mode function 200 or test mode function 300, is activated by user interface 112, results obtained from screenshot verification program 120 are stored within storage device 116 as master screenshot 122 or test screenshot 124, respectively. Upon completion, screenshot verification program 120 determines success or failure of a test scenario, such as test scenario 118, based on whether a webpage, application, or feature is working as designed.")
perform image matching techniques to compare the screenshot and the reference image (Hey, Fig. 3, and [0036-0046])
Regarding Claim 3, Hey in view of Liu teaches the system of claim 1. Hey teaches wherein the processor is further configured to: generate a test report for the changed DOM based on the comparison, the test report indicating a similarity value for the screenshot and the reference image (Hey, [0024] "An example embodiment of the invention is depicted and discussed in FIG. 4A-4D. FIG. 4A, example master screenshot 400, depicts a potential master screenshot 122 of a webpage. FIG. 4B, example test screenshot 420, depicts a potential test screenshot 124 of an updated webpage and a pass report. FIG. 4C, example test screenshot 440, depicts a different possible test screenshot of another updated webpage. FIG. 4D, example failure report 460, depicts a failure report." [0042-0046] "In step 312, test mode function 300 compares the master page objects with the test page objects. A comparison between master screenshot 122 and test screenshot 124 can be accomplished through various standard comparison techniques, such as image processing (e.g., pixelation, histogram, and red, green, blue (RGB) color model comparison). Image processing refers to the reading of an image by a computing device for the purpose of analyzing image characteristics. An image can be defined as a set of points (x,y) and an associated amplitude (e.g., brightness) for each point. Pixelation deconstructs images into the smallest element of an image (i.e., pixel) for comparison. A histogram graphically represents the distribution of data in an image and displays the information in categories for analysis or comparison. RGB color model comparison, utilizes a grid to evaluate an image based on the RGB triplet (r,g,b) with the color ranges being 0 to 1. Test scenario 118 can set a level of tolerance dependent on which comparison method is selected to account for variances and minor changes (e.g., color changes between computing devices, font smoothing, browser variance). In some embodiments when scaling was necessary (see step 310), test mode function 300 compares master page objects in master screenshot 122 with a scaled version of test screenshot 124. In another embodiment, test mode function 300 compares master page objects in master screenshot 122 with test screenshot 124 without scaling applied.…In step 316, test mode function 300 provides a failure report. In one embodiment, a failure may result due to missing page objects, as depicted in FIG. 4C, example 440. In another embodiment, a failure may result due to results not meeting tolerances set in test scenario 118. In one embodiment of the invention, test mode function 300 displays failures by highlighting the changed page objects on test screenshot 124. In another embodiment, test mode function 300 displays failures in two separate screenshots with all the page objects displayed and differences are highlighted in the second image from test screenshot 124. In another embodiment, test mode function 300 displays only the portions of the page objects which fail. In yet another embodiment, test mode function 300 provides a report that may list text information about which page objects have failed. In another embodiment, test mode function 300 provides a text report that may display pass and failure status with respect to acceptable tolerance levels. In step 318, test mode function 300 provides a pass report. In one embodiment, test mode function 300 indicates a pass by displaying a single screenshot image, without page objects highlighted. In another embodiment, test mode function 300 indicates a pass displaying a message stating the test has passed. In another embodiment, test mode function 300 displays a text report with the page objects noting a pass status. In another embodiment, test mode function 300 displays a text report depicting the results of the comparison in regards to the tolerance levels.")
Regarding Claim 4, Hey in view of Liu teaches the system of claim 1. Hey teaches wherein the reference image is stored in a database (Hey, [0023] "Test mode function 300, explained in greater detail in regards to FIG. 3, performs the selection and capture of updated conditions identified by test scenario 118, placed in storage device 116 and stored as test screenshot 124. Updated conditions refer to changes made to initial conditions that may or may not result in variances between control data (e.g., master screenshot 122) and test data (e.g., test screenshot 124). Updated conditions include, but are not limited to, changes made to the format or content, contained or displayed, by a webpage. For example, updated conditions may include changes in the user environment (e.g., new browser version), changes to the structure, design, or format of a webpage, or changes in the website environment (e.g., server operating system, database). In embodiments of the present invention, test screenshot 124 is the data to be validated against the control data stored within master screenshot 122." and [0020] "Storage device 116 can be implemented with any type of storage device that is capable of storing data that may be accessed and utilized by computing device 110. Value structures for data structure instances may be stored to storage device 116. In one embodiment, storage device 116 resides on computing device 110. In other embodiments, storage device 116 may reside on another server or another computing device connected over network 130, provided that storage device 116 is accessible to computing device 110. Storage device 116 includes test scenario 118, master screenshot 122, and test screenshot 124.")
Claims 5-8 are medium claims reciting limitations similar to claims 1-4 respectively and are rejected under the same rationale.
Claims 9-12 are method claims reciting limitations similar to claims 1-4 respectively and are rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Feng et al. (US20230221847A1): discloses UI error detection with advanced similarity score matrices.
Kumar et al. (US20130004087A1): discloses methods for webpage regression testing using screenshots.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR DARWISH whose telephone number is (571)272-4779. The examiner can normally be reached 7:30-5:30 M-Thurs.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lewis Bullock can be reached on 571-272-3759. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.E.D./Examiner, Art Unit 2199
/LEWIS A BULLOCK JR/Supervisory Patent Examiner, Art Unit 2199
Prosecution Insights