Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the preliminary amendment filed on 6/10/2024 to the Application filed on 3/21/2024.
This application is a continuation of U.S. Patent Application 17/451809 filed on 10/21/2021.
Claims 1-20 are pending in the case. Claims 1, 10, and16 are independent claims.
Claim Interpretation
Claim 1 recites the limitation “one or more interactive UI elements are stored in the RPA object repository of the RPA system in response to determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository”.
Claim 6 recites the limitation “the identifying the one or more descriptors with the top N match scores is in response to determining that none of the match scores are a maximum match score”.
Each of the “in response to” clauses indicates that the associated limitations occur only when the criteria of these clauses are met. However, the present claims never affirmatively require such events to occur. The broadest reasonable interpretation of these limitations does not require these conditional steps to be performed. See Ex parte Schulhauser, 2013-007847 (PTAB 2016) (precedential) where the board held that when method steps are to be carried out only upon the occurrence of a condition precedent, the broadest reasonable interpretation holds that those steps are not required to be performed. As such, the limitations followed by “in response to” clauses do not appear to have patentable weight since they are contingent upon a condition occurring.
Examiner suggests positively reciting the criteria occurring prior to each of the “in response to” limitations. For example, in claim 1, examiner suggests “determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository; wherein the one or more interactive UI elements are stored in the RPA object repository of the RPA system in response to determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository”. For example, in claim 6, examiner suggests “[[,]] further comprising, determining that none of the match scores are a maximum match score and wherein the identifying the one or more descriptors with the top N match scores is in response to determining that none of the match scores are a maximum match score”
Claim Objections
Claims 6 are objected to because of the following informality:
Claim 6 recites the limitation “the top N match scores.” There is insufficient antecedent basis for this limitation since there is no earlier recitation of “top N match score”. Examiner suggest amending claim 6 to depend on claim 5.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 9-10, 16, and 20 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Hosbettu et al., U.S. Patent Application Publication No. 20170228119, filed on 3/23/2016 (hereinafter Hosbettu) in view of Masuda, U.S. Patent Application Publication No. 20100084849, filed on 10/2/2008 (hereinafter Masuda).
As for independent claim 1, Hosbettu discloses apparatus and method comprising
receiving a selection of one or more interactive UI (user interface) elements stored in an RPA (robotic process automation) object repository of an RPA system; and
(Hosbettu paragraph [0022]-[0026] discloses receiving a selection of interactive UI elements, user can right-click, click, text input, make radio button selection on screen elements that are stored in database tables of screen elements)
recording an RPA automation of actions performed on the one or more interactive UI elements stored in the RPA object repository,
(Hosbettu paragraph [0022]-[0026] discloses recording an RPA automation of actions, RPA system can capture user activities performed on the interface objects that are stored in database tables of screen elements)
wherein the one or more interactive UI elements are stored in the RPA object repository of the RPA system
(Hosbettu paragraph [0022]-[0026] discloses UI interactive elements, interface objects, that are stored in database tables of screen elements).
Hosbettu does not appear to explicitly disclose apparatus and method wherein the one or more interactive UI elements are stored in the RPA object repository of the RPA system in response to determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository. However, Masuda discloses apparatus and method wherein
the one or more interactive UI elements are stored in the object repository of the system in response to determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository
(Masuda paragraph [0016] discloses storing user interface elements when it is determined not to be a duplicate).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Masuda with Hosbettu for the benefit of being able to filter objects to be store to limit usage of storage space.
As for claim 9, limitations of parent claim 1 have been discussed above. Hosbettu discloses apparatus and method wherein
the one or more interactive UI elements comprises at least one of a button, a checkbox, or a text field
(Hosbettu paragraph [0022]-[0026] discloses UI interactive elements include button, radio buttons and text boxes).
As for claim 10, claim 10 reflects article of manufacture comprising computer executable instructions for implementing method in claim 1 and is rejected along the same rationale.
As for claim 16, claim 16 reflects article of manufacture comprising computer executable instructions for implementing method in claim 1 and is rejected along the same rationale.
As for claim 20, limitations of parent claim 16 have been discussed above. Claim 20 reflects article of manufacture comprising computer executable instructions for implementing method in claim 9 and is rejected along the same rationale.
Claims 2, 11, and 17 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Hosbettu in view of Masuda in view of Yoshizumi, U.S. Patent No. 9639073, filed on 4/23/2013 (hereinafter Yoshizumi).
As for claim 2, limitations of parent claim 1 have been discussed above. Yoshizumi discloses apparatus and method wherein determining that the one or more interactive UI elements are not duplicates of one or more descriptors associated with existing interactive UI elements in the RPA object repository comprises:
generating match scores for descriptors associated with existing interactive UI elements in the RPA object repository;
(Yoshizumi Col 7 Lines 58-67, Col 8 Lines 11-24 discloses generating element identification value and converting the value to hash value)
identifying the one or more descriptors from the descriptors associated with the existing interactive UI elements based on the match scores; and determining whether the one or more identified descriptors match the one or more interactive UI elements
(Yoshizumi Col 7 Lines 58-67, Col 8 Lines 11-24 discloses duplication detecting unit 42 determining identifying element identification of elements in already generated layout and determining whether the hash value based on element identification match the element identification of elements in candidate layout).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Yoshizumi with Hosbettu and Masuda for the benefit of having a system “for discriminating between combined results of a plurality of elements”, (Yoshizumi Col. 1 Lines 19-20).
As for claim 11, limitations of parent claim 10 have been discussed above. Claim 11 reflects article of manufacture comprising computer executable instructions for implementing method in claim 2 and is rejected along the same rationale.
As for claim 17, limitations of parent claim 16 have been discussed above. Claim 11 reflects article of manufacture comprising computer executable instructions for implementing method in claim 2 and is rejected along the same rationale.
Claims 3-5, 7, 12-14, and 18 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Hosbettu in view of Masuda in view of Yoshizumi in view of Bissacco et. al., U.S. Patent No. 8520949, filed on 6/22/2009 (hereinafter Bissacco).
As for claim 3, limitations of parent claim 2 have been discussed above. Bissacco discloses apparatus and method wherein
the match scores represent a match between the descriptors and the one or more interactive UI elements
(Bissacco Col 1 Lines 42-57, Col 2 Lines 34-52 discloses similarity score represents match a score representing how often a descriptor matches to other similar descriptors).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Bissacco with Hosbettu, Masuda, and Yoshizumi for the benefit of having a system “that reduce false matches and improve performance of an image matching process as compared to present methods”, (Bissacco Col 1 Lines 38-39).
As for claim 4, limitations of parent claim 2 have been discussed above. Bissacco discloses apparatus and method wherein generating match scores for descriptors associated with existing interactive UI elements in the RPA object repository comprises:
filtering the descriptors to generate the match scores.
(Biassacco Col 1 Lines 42-57, Col 2 Lines 34-52 discloses filtering descriptors with similarity score exceeding threshold).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Bissacco with Hosbettu, Masuda, and Yoshizumi for the benefit of having a system “that reduce false matches and improve performance of an image matching process as compared to present methods”, (Bissacco Col 1 Lines 38-39).
As for claim 5, limitations of parent claim 2 have been discussed above. Bissacco discloses apparatus and method wherein identifying the one or more descriptors from the descriptors associated with the existing interactive UI elements based on the match scores comprises:
identifying the one or more descriptors with a top N match scores, where N is any positive integer
(Bissacco Col 5 Lines 63-67, Col 6 Lines 1-8 discloses identifying descriptors that exceed a threshold number such that the descriptors have a sufficiently high score).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Bissacco with Hosbettu, Masuda, and Yoshizumi for the benefit of having a system “that reduce false matches and improve performance of an image matching process as compared to present methods”, (Bissacco Col 1 Lines 38-39).
As for claim 7, limitations of parent claim 2 have been discussed above. Hosbettu discloses apparatus and method comprising
one or more interactive UI elements (Hosbettu paragraph [0022]-[0026] discloses UI interactive elements, interface objects such as textbox, label, button, that are stored in database tables of screen elements).
Hosbettu does not explicitly disclose apparatus and method wherein determining whether the one or more identified descriptors match the one or more interactive UI elements comprises: applying a unified target algorithm to the one or more identified descriptors to determine whether the one or more identified descriptors match the one or more interactive UI elements. However, Bissacco discloses apparatus and method wherein determining whether the one or more identified descriptors match the one or more interactive UI elements comprises:
applying a unified target algorithm to the one or more identified descriptors to determine whether the one or more identified descriptors match the other elements
(Bissaco Col 1 Lines 17-27, Col 2 Lines 34-52 discloses applying algorithm to determine descriptors of an object matches descriptors of other objects).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Bissacco with Hosbettu, Masuda, and Yoshizumi for the benefit of having a system “that reduce false matches and improve performance of an image matching process as compared to present methods”, (Bissacco Col 1 Lines 38-39).
As for claim 12, limitations of parent claim 11 have been discussed above. Claim 12 reflects article of manufacture comprising computer executable instructions for implementing method in claim 3 and is rejected along the same rationale.
As for claim 13, limitations of parent claim 11 have been discussed above. Claim 13 reflects article of manufacture comprising computer executable instructions for implementing method in claim 4 and is rejected along the same rationale.
As for claim 14, limitations of parent claim 11 have been discussed above. Claim 14 reflects article of manufacture comprising computer executable instructions for implementing method in claim 5 and is rejected along the same rationale.
As for claim 18, limitations of parent claim 17 have been discussed above. Claim 18 reflects article of manufacture comprising computer executable instructions for implementing method in claim 7 and is rejected along the same rationale
Claims 6 and 15 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Hosbettu in view of Masuda in view of Yoshizumi in view of Bohn et al., U.S. Patent Application Publication No. 20040143508, filed on 1/22/2003 (hereinafter Bohn).
As for claim 6, limitations of parent claim 4 have been discussed above. Bohn discloses apparatus and method wherein
the identifying the one or more descriptors with the top N match scores is in response to determining that none of the match scores are a maximum match score.
(Bohn paragraph [0034] discloses if no match is found filtering items with highest similarity score).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Bohn with Hosbettu, Masuda, Yoshizumi, and Biassacco for the benefit of being able to return usable result to a user in case a match is not found.
As for claim 15, limitations of parent claim 14 have been discussed above. Claim 15 reflects article of manufacture comprising computer executable instructions for implementing method in claim 6 and is rejected along the same rationale
Claims 8 and 19 are rejected under AIA 35 U.S.C. 103 as being unpatentable over Hosbettu in view of Masuda in view of Yoshizumi in view of Vottaw et al., U.S. Patent Application Publication No. 20190130094, filed on 10/26/2017 (hereinafter Votaw).
As for claim 8, limitations of parent claim 2 have been discussed above. Hosbettu discloses apparatus and method comprising
receiving user input selecting the one or more interactive UI elements from the identified interactive UI elements
(Hosbettu paragraph [0022]-[0026] discloses receiving a selection of interactive UI elements, user can right-click, click, text input, make radio button selection on screen elements that are stored in database tables of screen elements).
Hosbettu does not explicitly disclose apparatus and method comprising receiving user input selecting a window of an application displayed on a display device; and automatically identifying interactive UI elements in the window of the application.
However, Votaw discloses apparatus and method wherein the one or more interactive UI elements are stored in the RPA object repository of the RPA system further by:
receiving user input selecting a window of an application displayed on a display device;
(Votaw paragraph [0058], [0059] discloses receiving user selection of application window, operation of user device application 122)
automatically identifying interactive UI elements in the window of the application
(Votaw paragraph [0059] discloses automatically identifying interactive UI (user interface) elements, analyze the user interface to identify a pertinent graphical input element such as text box).
Accordingly, it would have been obvious to person of ordinary skill in the art before the effective filing date of the claimed invention to combine Votaw with Hosbettu, Masuda, and Yoshizumi for the benefit of having a system to automatically identify elements such that user doesn’t have to manually select each element thus saving time.
As for claim 19, limitations of parent claim 17 have been discussed above. Claim 19 reflects article of manufacture comprising computer executable instructions for implementing method in claim 8 and is rejected along the same rationale
Conclusion
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEANETTE J PARKER whose telephone number is (571)270-3647. The examiner can normally be reached Mon-Fri 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fred Ehichioya can be reached at 571-272-4034. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEANETTE J PARKER/Primary Examiner, Art Unit 2179