Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to the application filed 12/10/2024.
Claims 1-20 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Butin et al (US 9405558, hereinafter “Butin”) in view of Bekmambetov et al (US 10430212 B1, hereinafter “Bekmambetov”).
Regarding claim 1, Butin teaches a method, comprising: determining, in response to at least one first input to a user interface of a computing system, that at least one first action is to be taken with respect to a first user interface (UI) element being displayed by the user interface (action pertaining to a GUI control on the screen) (col. 4, lines 35-65) ; determining, by the computing system, first data corresponding to the first UI element (coordinates of the UI control position) ; and generating, by the computing system, a script configured to: determine that first coordinates corresponding to the first coordinate data are being displayed on a screen of a computing device (comparing the screen GUI control position data to the guidance script), and based at least in part on the first coordinates corresponding to the first coordinate data, cause the computing device to take the at least one first action at first coordinates corresponding to a first location on the screen at which of the first coordinates are being displayed (operate the GUI control when match occurs) (col. 5, lines 12-27), (col. 6, lines 21-27) (col. 9, lines 5-52) (Fig. 1-9). While Butin mentions pixel/pixel data, Butin does not explicitly disclose pixels at UI screen elements. Bekmambetov teaches a method for recording, editing and reproduction of computer session which is similar to that of Butin. Bekmambetov further teaches using pixels in defining screen element attributes (col. 10, lines 10-44). Since both Butin and Bekmambetov teachings are directed to recording, editing, and replaying sequences of mouse/user interaction, it would have been obvious to one skill in the art to include Bekmambetov’s teaching in Butin’s interface. One would have been motivated to make such a combination to provide Butin’s system the ability to calculate/address specific data points.
Regarding claim 2, Butin in view of Bekmambetov teaches wherein the first pixel data identifies at least one color value and at least one screen location corresponding to the at least one color value (Butin, col. 5, lines 27-67; col. 9, lines 5-52) (Bekmambetov, col. 10, lines 10-44, col. 12, lines 10-19; col. 13, lines 45-63).
Regarding claim 3, Butin in view of Bekmambetov further teaches determining, by the computing system, a first coordinate corresponding to the at least one first input; wherein determining the first pixel data corresponding to the first UI element includes identifying at least one pixel of the user interface within a vicinity of the first coordinate (Butin, col. 5, lines 27-67; col. 9, lines 5-52) (Bekmambetov, col. 10, lines 10-44).
Regarding claim 4, Butin in view of Bekmambetov further teaches determining, in response to at least one second input to the user interface of the computing system, that at least one second action is to be taken with respect to a second UI element being displayed by the user interface, wherein the at least one second input indicates a textual dependency; and further generating, by the computing system, the script to further be configured to: determine first textual data associated with the script; determine second textual data by performing optical character recognition of second pixel data being displayed on the screen of the computing device; determine that the first textual data corresponds to the second textual data; and based at least in part on determining the first textual data corresponds to the second textual data, cause the computing device to take the at least one second action at second coordinates corresponding to a second location on the screen at which at least one element corresponding to the first textual data is being displayed (Butin, line 11 of col. 5 to line 27 of col. 6; line 31 of col. 7 to line 45 of col. 8; Fig. 2-5) (Bekmambetov, col. 10, lines 10-44).
Regarding claim 5, Butin in view of Bekmambetov further teaches wherein the user interface is rendered by a browser (Bekmambetov, col. 2, lines 37-60, col. 10, lines 10-44, Fig. 2-6). It would have been obvious to one skill in the art to include Bekmambetov’s teaching in Butin’s interface. One would have been motivated to make such a combination to provide Butin’s system the ability to operate in both local and web environment.
Regarding claim 6, Butin in view of Bekmambetov further teaches wherein the script includes a uniform resource locator (URL) corresponding to a web page to be initially rendered by the browser (Bekmambetov, col. 2, lines 37-60, col. 10, lines 10-44, Fig. 2-6).
Regarding claim 7, Butin in view of Bekmambetov further teaches wherein the method is performed by a component of the browser (Bekmambetov, col. 2, lines 37-60, col. 10, lines 10-44, Fig. 2-6).
Regarding claim 12, Butin in view of Bekmambetov further teaches determining a first number of the first pixels corresponding to the first pixel data exceeds a threshold value (Butin, line 11 of col. 5 to line 58 of col. 6) (Bekmambetov, col. 10, lines 10-44).
Claims 8-11 and 13-20 are rejected based on similar rationale.
Butin further teaches processor and computer readable medium recited in claim 16 (Butin, col.4, lines 10-25, claim 12)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
White (US 20130132833), Zhao (US 8312428), and Butin (US 20100205529) teach recording and replaying computer interactions which relates to the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kieu Vu whose telephone number is (571) 272-4057. The examiner can normally be reached on Monday - Thursday (6:30am - 4:30pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached on (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIEU D VU/Supervisory Patent Examiner, Art Unit 2171
