DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is responsive to the Application filed on 7/10/2024. Claims 1-15 are pending in the case.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 8-9, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Hailpern et al. (US 20180329726 A1, hereinafter Hailpern) in view of Milov et al. (US 20090103769 A1, hereinafter Milov)
As to independent claim 1, Hailpern teaches a method for recording and playback, comprising:
recording an operation of a user in a first device based on an operation recording instruction to obtain a user operation record (“The user-activatable element programming engine 102 records (at 302) user actions in multiple different environments, including environments provided by multiple applications, for example. The recording can be initiated in response to a user input, such as user selection of the record element 110 (FIG 1).”);
acquiring environment data of the first device (“Initiation of the recording starts a record mode, in which user actions of made with respect to different applications can be monitored and recorded. During use of the applications, different ones of the applications can be in focus at different times.” Paragraph 0033-0034); and
performing user operations of the user operation record sequentially in a second device based on an operation playback instruction (“At system 2, a user action replay engine 502 (similar to the user action replay engine 104 discussed above) can use the template 500 to cause the customized user-activatable element 112 to be displayed in a display 504 in system 2 as 506, so that a user at system 2 can activate the customized user-activatable element 506 to replay the associated recorded user actions at system 2.” Paragraph 0043).
Hailpern does not appear to expressly teach wherein the performing the user operations of the user operation record comprises, for one of the user operations:
determining a corresponding target control in the second device based on the environment data of the first device and environment data of the second device, and
performing a respective user operation based on the target control.
Milov teaches wherein the performing the user operations of the user operation record comprises, for one of the user operations:
determining a corresponding target control in the second device based on the environment data of the first device and environment data of the second device (“The cognitive control framework (CCF) system of embodiments of the present invention performs image analysis and object detection processing on display data from the GUI of the application program. The CCF system includes comparing an image captured during a recording phase (called IR) to the corresponding image captured during the playback phase (called IP). One task of the system is to detect an object in the IR to which the user applied an action, find the corresponding object in the IP, and continue progress on the execution path of the execution scenario by applying the action to the detected object. These steps may be repeated for multiple objects within an image, and may be repeated across multiple pairs of IRs and IPs over time.” Paragraph 0021,0018, “playback might be executed in different operating environments such as different screen resolutions, different visual schemes, different window layouts, and so on.” Paragraph 0022); and
performing a respective user operation based on the target control (“An object that the user has applied an action to may be called an "object of action.", paragraph 0021).
Accordingly, it would have been obvious to a person of skill in the art before the effective filing date of the claimed invention to modify the method of Hailpern to comprise wherein the performing the user operations of the user operation record comprises, for one of the user operations: determining a corresponding target control in the second device based on the environment data of the first device and environment data of the second device, and performing a respective user operation based on the target control. One would have been motivated to make such a combination to “introduce only minimal disturbances and delay of application execution during playback.” Milov [0022] last sentence.
As to dependent claim 2, Hailpern teaches the method according to claim 1, Hailpern further teaches wherein the environment data includes:
device-related data, application-related data, user-related data, or user operation-related data (“Initiation of the recording starts a record mode, in which user actions of made with respect to different applications can be monitored and recorded. During use of the applications, different ones of the applications can be in focus at different times.” Paragraph 0033-0034).
Hailpern does not appear to expressly teach wherein the user operation-related data includes:
a page screenshot, an XML layout file of a page, or a control screenshot of an operation object control.
Milov teaches wherein the user operation-related data includes:
a page screenshot, an XML layout file of a page, or a control screenshot of an operation object control (“This input data is captured and stored by the CCF system. Next, at block 108, the display data may be captured (e.g. screen shots are taken). In one embodiment, the display data may captured only when user input has been received by the application program. The display data is also saved. At block 110, the data captured during blocks 106 and 108 may be analyzed and saved. These processes may be repeated a plurality of times. The result of the processing of the recording component comprises an execution scenario 112 for the application program being processed by the system. In one embodiment, the execution scenario comprises a script containing Extended Markup Language (XML) tags.” Paragraph 0018).
Accordingly, it would have been obvious to a person of skill in the art before the effective filing date of the claimed invention to modify the method of Hailpern to comprise wherein the user operation-related data includes: a page screenshot, an XML layout file of a page, or a control screenshot of an operation object control. One would have been motivated to make such a combination to “introduce only minimal disturbances and delay of application execution during playback.” Milov [0022] last sentence.
Claims 8-9 and 15 are substantially the same as claims 1-2 and are therefore rejected under similar rationale as above.
Allowable Subject Matter
Claims 3-7 and 10-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Carmi US 20140189576 A1 teaches automatically replay a session, identify a screen displayed by an application by relating a screenshot of the screen to screenshots included in a recorded session, determine an interaction to be performed (e.g., a click or a selection in a menu) based on data in the recorded session and further determine the next screen to be presented following the even
Li et al. US 20160349928 A1 teaches generating summary of activity on computer GUI
Lin et al. US 20120131456 A1 teaches techniques for capture and playback of user-performed GUI-based (Graphical User Interface) tasks across multiple GUI-based applications.
Guttman et al. US 20110191676 A1 teaches cross-browser interactivity recording, playback, and editing.
Macbeth et al. US 20070299631 A1 teaches logging user actions withing activity context. These logged actions can be aggregated, synchronized and/or shared between multiple users and/or devices.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHELET SHIBEROU whose telephone number is (571)270-7493. The examiner can normally be reached Monday-Friday 9:00 AM-5:00 PM Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHELET SHIBEROU/Primary Examiner, Art Unit 2171
Prosecution Insights