Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/09/2025 is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 18-20 recite the limitation "The system" in the first line of each claim. There is insufficient antecedent basis for this limitation in the claim which makes it unclear what is considered “The system”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 12-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Young (US 20200094398 A1) in view of Lee (US 20260051323 A1).
Regarding claim 1, Young discloses A method comprising:
receiving, at an autonomous mobile robot, a request for the robot to remotely control a function of an electronic device comprising a display screen [Young ¶ 0040 "Robot 600 may be configured to be taught by a user, through voice commands and local images, that an object comprises an object of interest." and ¶ 0042 "FIG. 7 illustrates a robot 700 monitoring the state of an object of interest 710 according to an embodiment of the invention. In the example of FIG.7, object of interest 710 is an appliance (a washing machine), and robot 700 determines the state of a laundry job in progress." The robot is configured by the user to monitor the state of washing machine that includes a display screen showing the state of the laundry job as shown in Figure 7.];
and in response to a determination that the robot's location comprises the optimal location, then processing the request [Young ¶ 0043 "The state of object of interest 710 may be or include a finished task, or a problem that requires user attention. Robot 700 may determine a priority for alerting the user based on the type of task that has finished, or otherwise progressed to a different stage, or on the urgency or type of problem that needs to be solved." The robot will alert the user that the laundry is finished once it is able to properly view the display.].
Young does not teach determining, by the autonomous mobile robot and based on one or more images of the display screen captured by a camera of the robot, whether a current location of the robot comprises an optimal location to remotely control the function of the electronic device and in response to a determination that the robot's location does not comprise the optimal location, then adjusting, by the autonomous mobile robot, a position of the robot.
However, in a related field of invention, Lee does teach determining, by the autonomous mobile robot and based on one or more images of the display screen captured by a camera of the robot, whether a current location of the robot comprises an optimal location to remotely control the function of the electronic device [Lee ¶ 0117 " In order for the interactive care robot 100 to more accurately recognize the user's condition or surrounding situation and capture images, the operation controller 217 may be used to change the location or capture angle of the camera."]; and one or more of:
in response to a determination that the robot's location does not comprise the optimal location, then adjusting, by the autonomous mobile robot, a position of the robot [Lee ¶ 0120 " After capturing an image of the user, if a recognition rate of the user's pose is equal to or less than a predetermined threshold, the interactive care robot 100 may improve the pose recognition rate by controlling the camera controller 218, which may change the capture direction of the camera by controlling at least one of the tilting unit, the panning unit, and the lifting unit."];
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine the object of interest monitoring robot as taught by Young with using the repositioning robot to get a better image as taught by Lee in order to get a better view of the display and perform more accurate monitoring.
Regarding claim 2, Young and Lee teach claim 1. Lee further teaches wherein the optimal location comprises a location at which the one or more images of the display screen meet one or more criteria for one or more visual parameters of a view of the display screen [Lee ¶ 0120 " After capturing an image of the user, if a recognition rate of the user's pose is equal to or less than a predetermined threshold, the interactive care robot 100 may improve the pose recognition rate by controlling the camera controller 218, which may change the capture direction of the camera by controlling at least one of the tilting unit, the panning unit, and the lifting unit."].
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine the object of interest monitoring robot as taught by Young with using the repositioning robot to get a better image as taught by Lee in order to get a better view of the display and perform more accurate monitoring.
Regarding claim 3, Young and Lee teach claim 2. Lee further teaches wherein the one or more visual parameters comprise one or more of (1) a reflection from the display screen (2) an occlusion of the display screen and (3) an angle at which the camera captures an image of the display screen [Lee ¶ 0120 " After capturing an image of the user, if a recognition rate of the user's pose is equal to or less than a predetermined threshold, the interactive care robot 100 may improve the pose recognition rate by controlling the camera controller 218, which may change the capture direction of the camera by controlling at least one of the tilting unit, the panning unit, and the lifting unit."].
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine the object of interest monitoring robot as taught by Young with using the repositioning robot to get a better image as taught by Lee in order to get a better view of the display and perform more accurate monitoring.
Regarding claim 4, Lee and Young teach claim 1. Lee further teaches wherein adjusting, by the autonomous mobile robot, a position of the robot comprises:
determining, by the autonomous mobile robot, a location of the robot relative to the display screen [Lee ¶ 0016 "FIG. 6 illustrates an example in which the interactive care robot captures an image of the user 20 from a first position 601 and then moves to a second position 602 to capture another image of the user 20 in order to perform a more accurate situation analysis based on the captured images."];
determining, by the autonomous mobile robot, one or more candidate optimal locations from which to remotely control the functionality of the electronic device [Lee ¶ 0016 "FIG. 6 illustrates an example in which the interactive care robot captures an image of the user 20 from a first position 601 and then moves to a second position 602 to capture another image of the user 20 in order to perform a more accurate situation analysis based on the captured images."]; and
navigating, by the autonomous mobile robot, to at least one of the one or more candidate optimal locations [Lee ¶ 0016 "FIG. 6 illustrates an example in which the interactive care robot captures an image of the user 20 from a first position 601 and then moves to a second position 602 to capture another image of the user 20 in order to perform a more accurate situation analysis based on the captured images."].
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention with a reasonable expectation of success to combine the object of interest monitoring robot as taught by Young with using the repositioning robot to get a better image as taught by Lee in order to get a better view of the display and perform more accurate monitoring.
Regarding claim 5, Young and Lee teach claim 1. Young further teaches wherein processing the request comprises:
capturing an image of content currently displayed on the display screen [Young ¶ 0042 " Robot 700 may use local images, as depicted, or may use remote images received from another device, such as a WiFi-connected camera that is installed near the object of interest 710.",];and
determining, based on the user request and the captured image, a task execution plan for fulfilling the user request [Young ¶ 0042 "FIG. 7 illustrates a robot 700 monitoring the state of an object of interest 710 according to an embodiment of the invention. In the example of FIG.7, object of interest 710 is an appliance (a washing machine), and robot 700 determines the state of a laundry job in progress." The robot is configured by the user to monitor the state of washing machine that includes a display screen showing the state of the laundry job as shown in Figure 7.].
Regarding claim 12, Young and Lee teach claim 1. Young further teaches wherein the electronic device comprises a television [Young ¶ 0047-0050 "upon determining that the user enters, turning on the TV and forwarding a user profile to the TV" and Figure 8].
Regarding claim 13-16 and 20, Young discloses An autonomous mobile robot comprising: [Young ¶ 0033 “Robot 110 is configured to autonomously move around in location 100.”]
a camera [Young ¶ 0031 “Robot 110 comprises camera 112”]; and
one or more non-transitory computer readable storage media storing instructions [Young ¶ 0071 “A computer may be any processor in communication with a memory.”], and one or more processors coupled to the one or more non-transitory computer readable storage media and operable to execute the instructions to [Young ¶ 0071 “A computer may be any processor in communication with a memory. The memory may be any suitable processor-readable storage medium, such as random-access memory (RAM), read-only memory (ROM), magnetic or optical disk, or other non-transitory media suitable for storing instructions for execution by the processor.”]:
The remaining limitations have been examined with respect to the method in claims 1-3, 5, and 12. The method taught/disclosed in claims 1-3, 5, and 12 can clearly perform on the system of claims 13-16 and 20. Therefore, claims 13-16 and 20 are rejected under the same rationale.
Allowable Subject Matter
Claims 6-11 and 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and resolve the 112(b) issues for claims 18 and 19. The limitations not found are “determining, by a VLM of the robot, a graph structure representing a set of display-screen UI states; and determining a navigation of the graph structure from an initial state corresponding to the captured image to a final state corresponding to fulfilling the user request.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPHINE RICH whose telephone number is (571)272-6384. The examiner can normally be reached M-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Browne can be reached at (571) 270-0151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.E.R./Examiner, Art Unit 3666
/SCOTT A BROWNE/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights