Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/16/2024 is being considered by the examiner.
Drawings
The drawings filed on: 03/15/2024 are accepted.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 and 4-7 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jiang et al (US Application: US 2022/0283769, published: Sep. 8, 2022, filed: Aug. 17, 2021).
With regards to claim 1, Jiang et al teaches a method for confirming event status in robotic process automation through images, applied in an operation system (OS), comprising:
step a) initiating a robotic process automation (RPA) software in the OS (Abstract, paragraph 0020: a robotic process automation is implemented on a device having an operating environment/system);
step b) executing a pre-recorded event script by the RPA software (paragraph 0018: a script is executed as part of RPA logic/software );
step c) obtaining a pre-stored origin image associated with the event script, wherein the origin image comprises a target image of a specific geographic range on a displaying screen of the OS (paragraphs 0027-0030: a pre stored image is captured for an event script representing an event, where the image is then used as a target image located within a displayed screen);
step d) automatically capturing a screenshot of the geographic range to obtain a current status image by the RPA software (paragraph 0027-0030: a current status is obtained by capturing a screen shot of a device);
step e) comparing the current status image with the origin image (paragraphs 0027-0030: the captured screenshot is compared to the origin image);
step f) capturing a screenshot of the geographic range again to obtain another current status image and re-executing the step e) when the current status image is determined to be unmatched with the origin image (paragraph 0034: screenshots are continually captured until there is a match/recognition that occurs. It is noted that this claim limitation is considered a contingent limitation that can be optional based upon when it is assessed that there is no match between current status image and origin image); and
step g) executing a next step of the event script by the RPA software when the current status image is determined to be matched with the origin image (paragraph 0027-0030, 0034: when the current image is determined to match the origin image, an action can be subsequently performed).
With regards to claim 4. The method in claim 1, Jiang et al teaches wherein the target image indicates a specific status of a procedure corresponding to the event script, and the geographic range is a displaying position that the specific status being shown on the displaying screen (paragraphs 0027-0030 and 0034: the target image represents a particular condition/status that has a corresponding action in the event script and a displayed position from the current screen capture is checked against the target image for a match/recognition).
With regards to claim 5. The method in claim 1, Jiang et al teaches wherein the step c) comprises obtaining a first origin image and a second origin image associated with the event script, wherein the first origin image comprises a first target image of a first geographic range on the displaying screen, the second origin image comprises a second target image of a second geographic range on the displaying screen, and the second geographic range is different from the first geographic range; wherein, the step d) comprises automatically capturing a first current status image from the first geographic range and capturing a second current status image from the second geographic range (paragraphs 0027- 0030: one or more target images (interpreted as the claimed ‘first origin’ and ‘second origin’ image)) are captured and each can correspond to a specific location on a screen through automatic capture using script creation).
With regards to claim 6. The method in claim 5, Jiang et al teaches wherein the step e) comprises: step e21) comparing the first current status image with the first origin image that are corresponding to the first geographic range; step e22) further comparing the second current status image with the second origin image that are corresponding to the second geographic range if the first current status image is determined to be matched with the first origin image; step e23) executing the step f) if the second current status image is determined to be unmatched with the second origin image; and step e24) executing the step g) if the second current status image is determined to be matched with the second origin image (paragraphs 0029, 0030, 0031 and 0034: current image is compared to a first origin image such as ‘Current Temperature’ to determine if there is a match. If no match the process continuously checks again for known origin image matches until there is a match (such as a match of a second current image against a second origin image) (such as for a second origin image of a ‘warning icon’)).
With regards to claim 7. The method in claim 6, Jiang et al teaches wherein the step e) further comprises: step e25) determining that an error occurs while the execution of the event script if the first current status image is determined to be unmatched with the first origin image; and step e26) issuing an alarming message after the step e25) (the limitations of claim 7 in step e25 are contingent limitations specific to when the first current status image is determined to be unmatched with the first. However in claim 6, the interpretation was that the first current status image is determined to be matched with the first origin image, and thus, the limitations in claim 7 are not required to be performed. In other words, due to the earlier condition precedent being met in claim 6 (for the first current status image being matched to the first origin image) the BRI of this claim (claim 7) does not encompass the limitations of claim 7 to be performed. Thus, claim 7 is rejected under similar rationale as claim 6 above since it incorporates the limitations of claim 6).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 and 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al (US Application: US 2022/0283769, published: Sep. 8, 2022, filed: Aug. 17, 2021) in view of Fan et al (US Application: 20090148042, published: Jun. 11, 2009, filed: Dec. 5, 2007).
With regards to claim 2. The method in claim 1, Jiang et al teaches step e12) respectively executing a … comparison algorithm to the current status image and the origin image … [between] … the current status image …[and] … of the origin image; and step e13) … determining whether the current status image is matched with the origin image, as similarly explained in the rejection of claim 1, and is rejected under similar rationale.
However Jiang et al does not expressly teach … wherein the step e) further comprises:
step e11) respectively performing a binarization operation to the current status image and the origin image; step e12) respectively executing a pixel comparison algorithm to the current status image and the origin image after the binarization operation to respectively obtain effective pixels of the current status image and effective pixels of the origin image; and
step e13) computing a similarity proportion of the current status image and the origin image based on the effective pixels of the current status image and the effective pixels of the origin image and determining whether the current status image is matched with the origin image in accordance with the similarity proportion.
Yet Fan et al teaches … wherein the step e) further comprises:
step e11) respectively performing a binarization operation to [a second image] and [a first image]; step e12) respectively executing a pixel comparison algorithm to the [the second image] and [the first image] after the binarization operation to respectively obtain effective pixels of the [the second image] and effective pixels [of the first image] ; and step e13) computing a similarity proportion of the [second image] and [the first image] based on the effective pixels of [the second image] and the effective pixels of [the first image] and determining whether [the second image] is matched with [the first image] in accordance with the similarity proportion (paragraph 0017 and 0023: binarization is first performed on image data having second and first character image data and then the second is compared against the first character image to determine a percentage (ratio) of similarity of being greater than a threshold amount to assess that they sufficiently match or else a match/group is not implemented).
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to have modified Jiang et al’s ability to compare two images ( a second image as a current status image and a first image as an origin image) to assess /determine a match , such that the match determination between the second and first images would have used binarization and then similarity assessment for match determination, as taught by Fan et al. The combination would have allowed a level of tolerance when assessing matches between image data.
With regards to claim 3. The method in claim 2, the combination of Jiang et al and Fan et al teaches wherein the step e13) further comprises: step e131) dividing the effective pixels of the current status image by the effective pixels of the origin to obtain a calculation result and multiplying the calculation result by 100% to obtain the similarity proportion; step e132) determining that the current status image is matched with the origin image if the similarity proportion is greater than or equal to a threshold; and step e133) determining that the current status image is unmatched with the origin image if the similarity proportion is smaller than the threshold, as similarly explained in the rejection of claim 2 (as explained the Jiang et al’s matching assessment was modified with Fan et al’s ability to further match through similarity proportion assessment (percentage) of amount of match being greater than or equal to at least a particular amount in order to determine whether there is a match or not), and is rejected under similar rationale
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al (US Application: US 2022/0283769, published: Sep. 8, 2022, filed: Aug. 17, 2021) in view of Moorer et al (US Application: US 2007/0260713, published: Nov. 8, 2007, filed: Mar. 15, 2007) .
With regards to claim 8. The method in claim 1, Jiang et al teaches further comprising following steps before the step a): step a01) initiating the RPA software in the OS; step a02) accepting an operation through a human-machine interface by the OS; step a03) recording a tracing coordinate of the human-machine interface and an event being triggered by the human-machine interface to be the event script by the RPA software (paragraph 0027: a user can provide input operation to record event/action associated with each event script)
However Jiang et al does not expressly teach .. and step a04) setting a script schedule of the event script; wherein the step b) comprises automatically executing the event script by the RPA software based on the script schedule.
Yet Moorer et al teaches and step a04) setting a script schedule of the event script; wherein the step b) comprises automatically executing the event script by … [operating software ] … based on the script schedule (paragraph 0034: an interface includes an ability to schedule actions/scripts for automatic execution (such as automatic execute of particular time/day(s)).
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to have modified Jiang et al and Moorer et al’s ability to allow a user to configure a script for execution, such that the script could have been further configured for being automatically executed according to a schedule, as taught by Moorer et al. The combination would have allowed Jiang et al to have allowed more flexibility and control of how actions are executed.
Claim(s) 9 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al (US Application: US 2022/0283769, published: Sep. 8, 2022, filed: Aug. 17, 2021) in view of Kaye (US Application: US 2010/0042682, published: Feb. 18, 2010, filed: Aug. 14, 2009).
With regards to claim 9. The method in claim 1, Jiang et al teaches wherein the event script is executed to implement …., and the target image …(paragraphs 0027-0030 and 0034: one or more target images are captured and each can correspond to a specific location on a screen through automatic capture using script creation. Screenshots are then continually captured until there is a match/recognition that occurs using script execution and subsequent action can be performed )
However Jiang et al does not expressly teach … a file downloading procedure under a closed system, and the target image is an image indicating that a file is downloaded completely.
Yet Kaye teaches a file downloading procedure under a closed system, and … an image indicating that a file is downloaded completely (paragraph 0027 and 0031, Fig. 8: an arrow ‘connected’ status image (ref 816) indicates an application has been downloaded completely to establish synchronization communications between 814 and 818. Also shown in Fig. 8, is ref 806 which also displays a normal network connection state displayed as ‘you’re connected’).
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to have modified Jiang et al’s ability to execute an event script to perform an action/procedure and also obtain one or more target images to check desired status(es) such that the images could have each included network status and/or download indication, as taught by Kaye. The combination would have allowed the ability to inform a user of network status associated with an application being downloaded.
With regards to claim 11. The method in claim 5, the combination of Jiang and Kaye teaches wherein the step a) is executed to implement a file downloading procedure through website, the first target image is an image indicating a normal network connection, and the second target image is an image indicating that a file is downloaded completely, as similarly explained in the rejection of claim 9, and is rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Petrescu et al (US Patent: 12159101): This reference teaches a form filling RPA robot that mimics computing events that occur when a human operator performs the respective operation(s).
Goyal et al (US Application: US 2023/0168906): This application teaches comparing recordings in an RPA system in view of a threshold to decide whether steps are sufficiently similar to be considered equivalent.
Ganesan et al (US Patent: 11537416): This reference teaches detecting and handling new process scenarios by monitoring UI activity.
Balasubramanian et al (US Application: US 2020/0050983): This reference teaches capturing and generating a process workflow and comparing workflows of different users.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILSON W TSUI whose telephone number is (571)272-7596. The examiner can normally be reached Monday - Friday 9 am -6 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Queler can be reached at (571) 272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILSON W TSUI/Primary Examiner, Art Unit 2172
Prosecution Insights