DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims in Consideration
Claims 1-20 are pending in this application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/26/2024 was filed before the first office action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejection Notes
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-9, are rejected under 35 U.S.C. 102a1 as being anticipated by Ripa (US 20210109722 A1, published: 4/15/2021).
Claim 1: Ripa teaches a method, comprising:
displaying, by a computing device, a first web page retrieved using a first uniform resource locator (URL), the first web page including at least a first user interface (UI) element (the label displayed on a button of a web page can be found by parsing the HTML document associated with the respective web page [Ripa, 0063]);
in response to a user providing a first input corresponding to the first UI element, storing first data representing the first URL and second data identifying the first UI element, the second data being stored in association with the first data (a non-transitory computer-readable medium stores instructions which, when executed by at least one hardware processor of a computer system configured to expose a target user interface (UI) comprising a plurality of UI elements, cause the computer system, in response to receiving a user input indicating a robotic process automation (RPA) activity, to expose an activity configuration UI enabling a user to configure parameters of the RPA activity [Ripa, 0006]. Once the user has selected a type of activity (step 104 in FIG. 4), in a step 106 some embodiments expose an activity configuration UI structured according to the type of the respective activity (e.g., mouse click, data scraping, etc.), and display the respective configuration UI in a context of other members of the activity sequence [Ripa, 0042]. Encodings of characteristic data may include the characteristic data itself and/or other representations of such data, for instance an indicator of a network location (e.g., URL, network address) where the respective data can be accessed remotely [Ripa, 0072]);
in response to the user providing a second input corresponding to the first UI element, retrieving a second web page using a second URL associated with the first UI element, the second web page including at least a second UI element (in a step 108, application 46 may receive and process the user input indicating the selected target element, for instance by calling certain OS functions to detect the mouse click and identifying the clicked UI element [Ripa, 0044]; Examiner's Note: as illustrated in FIG. 3, if a user selects with a first input, window 60b, that window will then be displayed, offering the user a chance now to select an element on that page/window); and
generating a sharable record including at least the first data and the second data, the sharable record being configured to be accessed by a browser to identify, based on the second data being associated with the first data, the first UI element on the first web page when the first web page is rendered by the browser (for each activity/automation step, authoring application 46 may output to RPA script 50 an indicator of the respective activity (e.g., click, type into, etc.) and element-characteristic features determined in step 112 for the respective target element and user-facing label. Such data may include, among others, a code label, an element text, and an element image for the respective UI elements [Ripa, 0072]; Examiner's Note: wherein URL addresses are shareable with all users who access a website, wherein said address links to said websites, and wherein each selectable element is likewise accessible to all authorized users).
Claim 2: Ripa further teaches the method of claim 1, further comprising: in response to the user providing a third input corresponding to the second UI element, storing third data representing the second URL and fourth data identifying the second UI element, the fourth data being stored in association with the third data, wherein: the sharable record further includes the third data and the fourth data, and the sharable record is further configured to be accessed by the browser to identify, based on the fourth data being associated with the third data, the second UI element on the second web page when the second web page is rendered by the browser ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the first claim, but adding additional inputs/data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim 3: Ripa further teaches method of claim 1. Ripa further teaches further comprising: prior to the user providing the first input and in response to the user providing a third input corresponding to the first UI element, displaying at least one third UI element corresponding to an option to include an indicator of the first UI element in the sharable record, wherein the first input corresponds to selection of the at least one third UI element ([Ripa, FIG. 3, 5, 6, 7, 12, 15, 16]; Examiner's Note: as illustrated each figure demonstrates selectable elements that can be selected by a user. Without said user interface, no selections could be made).
Claim 4: Ripa further teaches method of claim 1. Ripa further teaches wherein the first input and the second input are a same action selecting the first UI element ([Ripa, FIG. 3, 5, 6, 7, 12, 15, 16]; Examiner's Note: a user can select the same element again and again, with different inputs).
Claim 5: Ripa further teaches method of claim 1. Ripa further teaches wherein the first input and the second input are different actions selecting the first UI element ([Ripa, FIG. 3, 7, 12; Examiner's Note: as illustrated, each figure contains multiple elements, that can be selected in different orders, and being different actions).
Claim 6: Ripa further teaches method of claim 1, further comprising: in response to the user providing a third input corresponding to a third UI element of the first web page, storing third data representing the first URL and fourth data identifying the third UI element, the fourth data being stored in association with the third data, wherein: the sharable record further includes the third data and the fourth data, and the sharable record is further configured to be accessed by the browser to identify, based on the fourth data being associated with the third data, the third UI element on the first web page after identifying the first UI element on the first web page ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the first claim, but adding additional inputs/data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim 7: Ripa further teaches method of claim 1. Ripa further teaches further comprising: receiving a third user input indicating the user has finished identifying UI elements for the sharable record; and in response to receiving the third user input, generating the sharable record (once the design phase of automation is complete, RPA script 50 may be transmitted to script repository 15 and/or distributed to other RPA clients for execution (see e.g., FIG. 1) [Ripa, 0073]).
Claim 8: Ripa further teaches method of claim 1. Ripa further teaches further comprising: receiving user contact data corresponding to at least a second user; and sending, using the user contact data, a copy of the sharable record to the second user (the resulting RPA script 50 is then distributed to multiple other users and machines for execution [Ripa, 0038]).
Claim 9: Ripa further teaches method of claim 1, further comprising: receiving a third user input indicating that subsequently selected UI elements are to be included in the sharable record, wherein the storing of the first data and the second data in response to the first input is based at least in part on receipt of the third user input ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the first claim, but adding additional inputs/data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 10-20, are rejected under 35 U.S.C. 103 as being unpatentable over Ripa (US 20210109722 A1, published: 4/15/2021), in view of Fialkow (US 20190243882 A1, published: 8/8/2019).
Claim 10: Ripa further teaches method of claim 1. Ripa does not teach further comprising: including, in the sharable record, an indicator that a recipient browser is to process the sharable record using specialized code, the specialized code being configured to cause, based at least in part on the first data and the second data, at least one other element on the first web page to become un-selectable when the first web page is rendered by the recipient browser.
However, Fialkow teaches further comprising: including, in the sharable record, an indicator that a recipient browser is to process the sharable record using specialized code, the specialized code being configured to cause, based at least in part on the first data and the second data, at least one other element on the first web page to become un-selectable when the first web page is rendered by the recipient browser (the method includes enabling a user to edit a website page of the website building system; the page having regular components and overlapped and hidden components; and determining display instructions for the enabling for the overlapped and hidden components according to activation conditions, the activation conditions based on a user selected point on the page, activation rules and information on components of the page, the information including component proximity to or component interaction with the user selected point, z-order and at least one of: general relationships between the components on the page, information on the user and information on the system [Fialkow, 0044]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the input driven launching of a GUI invention of Ripa, to include the un-selectable element feature of Fialkow.
One would have been motivated to make this modification to distinguish selectable from un-selectable webpage elements. Firstly to elements that are meant only to be visible but not selectable (such as, for example, the current weather or time that may be displayed on a webpage) users do not have a need to select objects that are viewable only. Also, so that users may intuitively interact with a page the way the designers intended.
Claim 11: Ripa teaches a computing system comprising: at least one processor (Processor(s) 22 [Ripa, 0074, FIG. 19]); and at least one computer-readable medium encoded with instructions which, when executed by the at least one processor (Memory unit 24 may comprise volatile computer-readable media (e.g. dynamic random-access memory—DRAM) storing data/signals/instruction encodings accessed or generated by processor(s) 22 in the course of carrying out operations [Ripa, 0075, FIG. 19]), cause the computing system to:
display, by a computing device, at least a first user interface (UI) element corresponding to a sharable record, the sharable record including first data representing a first uniform resource locator (URL) corresponding to a first web page and second data identifying a second UI element on the first web page (the label displayed on a button of a web page can be found by parsing the HTML document associated with the respective web page [Ripa, 0063]. For each activity/automation step, authoring application 46 may output to RPA script 50 an indicator of the respective activity (e.g., click, type into, etc.) and element-characteristic features determined in step 112 for the respective target element and user-facing label. Such data may include, among others, a code label, an element text, and an element image for the respective UI elements. Encodings of characteristic data may include the characteristic data itself and/or other representations of such data, for instance an indicator of a network location (e.g., URL, network address) where the respective data can be accessed remotely [Ripa, 0072]; Examiner's Note: wherein URL addresses are shareable with all users who access a website, wherein said address links to said websites, and wherein each selectable element is likewise accessible to all authorized users);
in response to a user providing a first input selecting the first UI element, display, using the first URL, the first web page, the first web page including the second UI element and at least a third UI element (once the user has selected a type of activity (step 104 in FIG. 4), in a step 106 some embodiments expose an activity configuration UI structured according to the type of the respective activity (e.g., mouse click, data scraping, etc.), and display the respective configuration UI in a context of other members of the activity sequence [Ripa, 0042]); and
in response to detecting selection of the second UI element, take at least one first action (in a step 108, application 46 may receive and process the user input indicating the selected target element, for instance by calling certain OS functions to detect the mouse click and identifying the clicked UI element [Ripa, 0044]; Examiner's Note: as illustrated in FIG. 3, if a user selects with a first input, window 60b, that window will then be displayed, offering the user a chance now to select an element on that page/window).
Ripa does not teach cause, based at least in part on the second data, the third UI element to be un-selectable.
However, Fialkow teaches cause, based at least in part on the second data, the third UI element to be un-selectable (the method includes enabling a user to edit a website page of the website building system; the page having regular components and overlapped and hidden components; and determining display instructions for the enabling for the overlapped and hidden components according to activation conditions, the activation conditions based on a user selected point on the page, activation rules and information on components of the page, the information including component proximity to or component interaction with the user selected point, z-order and at least one of: general relationships between the components on the page, information on the user and information on the system [Fialkow, 0044]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the input driven launching of a GUI invention of Ripa, to include the un-selectable element feature of Fialkow.
One would have been motivated to make this modification to distinguish selectable from un-selectable webpage elements. Firstly to elements that are meant only to be visible but not selectable (such as, for example, the current weather or time that may be displayed on a webpage) users do not have a need to select objects that are viewable only. Also, so that users may intuitively interact with a page the way the designers intended.
Claim 20, sharing similar elements to claim 11, are likewise rejected.
Claim 12: The combination of Ripa and Fialkow, teaches the computing system of claim 11, wherein: the sharable record further includes third data representing a second URL corresponding to a third web page and fourth data identifying a fourth UI element on the third web page, and the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to take the at least one first action by: displaying the third web page, the third web page including the fourth UI element and at least a fifth UI element; causing, based at least in part on the fourth data, the fifth UI element to be un-selectable; and in response to detecting selection of the fourth UI element, taking at least one second action ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the independent claim, but adding additional inputs and data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim 13: The combination of Ripa and Fialkow, teaches the computing system of claim 12, wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to cause the second UI element to be un-selectable by: determining that the third web page has been rendered by a browser; determining that a third URL corresponding to the third web page matches the second URL included in the sharable record; and determining to cause the fifth UI element to be un-selectable based at least in part on the third URL of the third web page corresponding to the second URL included in the sharable record ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the independent claim, but adding additional inputs and data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim 14: The combination of Ripa and Fialkow, teaches the computing system of claim 11, wherein: the sharable record further includes third data representing the first URL and fourth data identifying the third UI element on the first web page, and the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to take the at least one first action by: causing the third UI element to be selectable; and causing the second UI element to be un-selectable ((A) Combining prior art elements according to known methods to yield predictable results; (B) Simple substitution of one known element for another to obtain predictable results [MPEP 2143]; Examiner's Note: this is merely an extension of the independent claim, but adding additional inputs and data. It is implicit in computing that a computer program may run its code again and again, to allow for many inputs to continually make selections of selectable elements/buttons/icons/etc.).
Claim 15: The combination of Ripa and Fialkow, teaches the computing system of claim 11. Ripa further teaches wherein: the third UI element is displayed on a first portion of the first web page; the second UI element is displayed on a second portion of the first web page; and causing the third UI element to be un-selectable comprises positioning an overlay over at least the first portion of the first web page ([Ripa, FIG. 3, 5, 6, 7, 12, 15, 16]; Examiner's Note: a user can select the same element again and again, with different inputs).
Claim 16: The combination of Ripa and Fialkow, teaches the computing system of claim 11. Ripa further teaches wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to: position an overlay over at least the second UI element and the third UI element on the first web page; and cause the second UI element to be selectable on the first web page by positioning the second UI element over the overlay (FIG. 11 shows an exemplary degree of overlap 67 according to some embodiments of the present invention, degree of overlap 67 determined as a proportion of one element intersecting the other element, or stated otherwise, how much of one element overlaps with the other. In such embodiments, two elements that do not intersect have zero overlap, whereas two elements wherein one element completely contains the other have 100% overlap. Some embodiments scores determined according to a degree of overlap to reliably identify labels for specific types of target elements (e.g., buttons) [Ripa, 0050]).
Claim 17: The combination of Ripa and Fialkow, teaches the computing system of claim 11. Fialkow teaches wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to cause the third UI element to be un-selectable by: configuring at least one attribute of the third UI element to cause the third UI element to be un-selectable (the method includes enabling a user to edit a website page of the website building system; the page having regular components and overlapped and hidden components; and determining display instructions for the enabling for the overlapped and hidden components according to activation conditions, the activation conditions based on a user selected point on the page, activation rules and information on components of the page, the information including component proximity to or component interaction with the user selected point, z-order and at least one of: general relationships between the components on the page, information on the user and information on the system [Fialkow, 0044]).
Claim 18: The combination of Ripa and Fialkow, teaches the computing system of claim 17. Fialkow teaches wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to configure the at least one attribute by: disabling a selection functionality of the third UI element (the method includes enabling a user to edit a website page of the website building system; the page having regular components and overlapped and hidden components; and determining display instructions for the enabling for the overlapped and hidden components according to activation conditions, the activation conditions based on a user selected point on the page, activation rules and information on components of the page, the information including component proximity to or component interaction with the user selected point, z-order and at least one of: general relationships between the components on the page, information on the user and information on the system [Fialkow, 0044]).
Claim 19: The combination of Ripa and Fialkow, teaches the computing system of claim 11. Ripa further teaches wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to: in response to receiving the first input, retrieve, with a web browser, the first web page using the first URL included in the sharable record (in a step 108, application 46 may receive and process the user input indicating the selected target element, for instance by calling certain OS functions to detect the mouse click and identifying the clicked UI element [Ripa, 0044]; Examiner's Note: wherein the input selectively opens a webpage).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SETH A SILVERMAN whose telephone number is (571)272-9783. The examiner can normally be reached Mon-Thur, 8AM-4PM MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Queler can be reached at (571)272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Seth A Silverman/Primary Examiner, Art Unit 2172
