DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Pursuant to communications filed on 28 May 2025, this is a First Action Non-Final Rejection on the Merits. Claims 1-11 are currently pending in the instant application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 28 May 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the Examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, Applicant provides the claim limitation including “output a control signal for controlling the robot in a predetermined section” and further wherein “the control unit move the robot to the teaching point and controls the robot in the predetermined section based on the control signal”, however, based on the currently provided claim language, it is unclear what the metes and bounds regarding the claimed “predetermined section” encompass, and therefore claim 1 is rendered indefinite. Specifically, based on the currently provided claim language, it is unclear if the claimed “predetermined section” correlates to a part of the robot, or alternatively a specified area within the environment relative to a teaching point, or some alternative and/or variation thereof, and as such claim 1 is rejected under this section. Accordingly, appropriate correction and/or clarification are earnestly solicited.
Regarding claims 2-9, these claims are either directly or indirectly dependent upon independent claim 1 and therefore are also rejected under this section for at least their dependency upon a rejected base claim. Accordingly, appropriate correction and/or clarification are earnestly solicited.
Regarding claim 10, Applicant provides the claim limitation including “outputting a control signal for controlling the robot in a predetermined section” and further wherein “the robot is controlled in the predetermined section based on the control signal”, however, based on the currently provided claim language, it is unclear what the metes and bounds regarding the claimed “predetermined section” encompass, and therefore claim 10 is rendered indefinite. Specifically, based on the currently provided claim language, it is unclear if the claimed “predetermined section” correlates to a part of the robot, or alternatively a specified area within the environment relative to a teaching point, or some alternative and/or variation thereof, and as such claim 10 is rejected under this section. Accordingly, appropriate correction and/or clarification are earnestly solicited.
Regarding claim 11, this claim is dependent upon independent claim 10 and therefore is also rejected under this section for at least its dependency upon a rejected base claim. Accordingly, appropriate correction and/or clarification are earnestly solicited.
Examiner notes wherein the claims have been addressed below in view of the prior art of record, as best understood by the Examiner, in light of the 35 USC 112(b), or second paragraph, rejections provided herein.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al (US 2025/0262760 A1, hereinafter Zhao) in view of Yanagawa et al (US 2011/0238215 A1, hereinafter Yanagawa).
Regarding claim 1, Zhao teaches a robot apparatus comprising:
a robot (Figure 4, robot 400), including an imaging unit (Figure 4, camera 450) and a control unit (Figure 4, controller 440), configured to operate at a plurality of teaching points based on an instruction provided by the control unit (Figures 4-10; at least as in paragraphs 0037, 0041, 0072 and 0074, wherein “The robot 400 is in communication with a controller 440, which controls movement of the robot 400 and also receives data from the robot 400. A camera 450 is mounted on the outer robot arm such that the camera 450 is fixed in orientation relative to the gripper 402” and further wherein “At the box 1014, the trained pose estimation neural network is used in inference mode for visual servoing control of the robot. This could include any of the control system architectures illustrated in FIGS. 7-9, where the trained visual pose estimation neural network 530 is used in a pure visual servoing control system (FIG. 7), or the neural network 530 is used in visual servoing control for preliminary positioning followed by compliance control for final positioning (FIG. 8), or the neural network 530 is used in an integrated visual servoing/compliance control system (FIG. 9)”);
a processing unit configured to execute visual servoing based on a reference image that is an image representing a goal state of the robot and a captured image acquired by the imaging unit driven to each of the plurality of teaching points and output a control signal for controlling the robot in a predetermined section (Figures 4-10; at least as in paragraphs 0037, 0041, 0054, 0059, 0063 and 0072-0074, wherein “At the box 1014, the trained pose estimation neural network is used in inference mode for visual servoing control of the robot. This could include any of the control system architectures illustrated in FIGS. 7-9, where the trained visual pose estimation neural network 530 is used in a pure visual servoing control system (FIG. 7), or the neural network 530 is used in visual servoing control for preliminary positioning followed by compliance control for final positioning (FIG. 8), or the neural network 530 is used in an integrated visual servoing/compliance control system (FIG. 9)” and further wherein “the software applications and modules of these computers and controllers are executed on one or more computing devices having a processor and a memory module configured for learning visual pose estimation in robotic operations. In particular, this includes a processor in the robot controller 440 along with the optional separate computer used for data collection in FIG. 4, the computer used for the training process of FIG. 5, and the robot controllers and optional other computers which execute the functions of the visual servoing and compliance controllers”); and
wherein in a case where the mode selection unit selects a first operation mode as the operation mode from the plurality of operation modes, the control unit moves the robot to the teaching point and causes the imaging unit to acquire the reference image (Figures 4-6 & 10; at least as in paragraphs 0036-0037 and 0039, wherein “the system of FIG. 4 is employed to collect data used for training a pose estimation neural network which, after training, will be used for visual servoing robot control. The data used for training is in the form of a plurality of images from the camera 450, with a robot pose recorded for each of the images. The robot pose which is recorded is the relative pose, in six degrees of freedom, of a tool center point with respect to a target pose”) Examiner notes wherein the offline data collection of images by the camera for training of a pose estimation (as detailed in at least Figure 4 of Zhao) is construed as a “first operation mode”., and
wherein in a case where the mode selection unit selects a second operation mode as the operation mode from the plurality of operation modes, the control unit moves the robot to the teaching point and controls the robot in the predetermined section based on the control signal (Figures 4-6 & 10; at least as in paragraphs 0041-0044, wherein “the images in the block 510 are taken from a variety of positions in the vicinity of the target pose, and under a variety of lighting conditions. The positions captured in the images may form a grid pattern or some other geometric or defined pattern surrounding the target pose, in which case the movement of the robot 400 and the triggering of the camera 450 to take each image may be defined in a control program” and further wherein “The block 520 contains the relative pose, recorded by the robot controller 440, corresponding with each image in the block 510. The relative pose defines the robot tool center point position relative to the target workpiece position, and was captured for each image”) Examiner notes wherein the learning of the pose estimation in a neural network (as detailed in at least Figures 5-6 of Zhao) is construed as a “second operation mode”.. That said, Zhao is silent specifically regarding wherein the robot apparatus specifically includes “a mode selection unit configured to select one of a plurality of operation modes as an operation mode of the robot at each of the plurality of teaching points”.
Yanagawa, in the same field of endeavor of teaching industrial robots/manipulators, teaches a programming method, apparatus and robot control system for teaching a robot to perform a task through teaching the robot to move and collect sensor data at different teaching points/positions associated with the task. Yanagawa further teaches wherein the robot may include a teach pendant that is connected to a robot controller of the robot, and further wherein said teach pendant may include a “mode selection key 32” for switching the robot between a plurality of operation modes (Figures 3 & 4; at least as in paragraphs 0036-0039 and 0057-0060, specifically as in at least paragraph 0057, wherein “when the mode selection key 32 in the teach pendant TP is operated by the operator, mode selection/execution process at S10 is performed. The mode selection/execution process allows execution of a teaching mode process at S20, sensing mode process at S30, or execution mode process at S40”). Therefore, it would have been obvious to one of ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Zhao to include Yanagawa’s teaching of providing the functionality of switching between a plurality of operating modes via a mode selection switch during programming (i.e. teaching) of said robot, since Yanagawa teaches wherein providing such functionality reduces the actual task cycle time by simplifying teaching tasks for the robot, thereby providing a more efficient and robust robot control system.
Regarding claim 2, in view of the above combination of Zhao and Yanagawa, Zhao further teaches wherein the plurality of operation modes farther includes a third operation mode, and
wherein in a case where the mode selection unit selects the third operation mode as the operation mode from the plurality of operation modes, the control unit moves the robot to the teaching point, operates the robot in the predetermined section under control based on the control signal, and causes the imaging unit to acquire the reference image while the robot is operating in the predetermined section (Figures 7-10; at least as in paragraphs 0054-0057, 0059-0061 and 0063-0072).
Regarding claim 3, in view of the above combination of Zhao and Yanagawa, Zhao further teaches wherein the control unit causes the imaging unit to acquire a plurality of the reference images at predetermined intervals while the robot is operating in the predetermined section (Figures 7-10; at least as in paragraphs 0054-0057, 0059-0061 and 0063-0072).
Regarding claim 4, in view of the above combination of Zhao and Yanagawa, Zhao further teaches wherein the plurality of operation modes further includes a fourth operation mode, and
wherein in a case where the mode selection unit selects the fourth operation mode as the operation mode from the plurality of operation modes, the control unit moves the robot to the teaching point, does not perform control based on the control signal, and does not cause the imaging unit to acquire the reference image (Figures 7-10; at least as in paragraphs 0054-0057, 0059-0061 and 0063-0072) .
Regarding claim 5, in view of the above combination of Zhao and Yanagawa, Zhao teaches the apparatus further comprising a storage unit (Figure 10, database 1006), wherein in a case where the reference image corresponding to the teaching point is not stored in the storage unit, the mode selection unit selects the first operation mode as the operation mode from the plurality of operation modes (Figures 7-10; at least as in paragraphs 0054-0057, 0059-0061 and 0063-0072).
Regarding claim 6, in view of the above combination of Zhao and Yanagawa, Zhao teaches the apparatus further comprising a storage unit (Figure 10, database 1006), wherein the mode selection unit selects the operation mode from the plurality of operation modes based on a program stored in the storage unit that controls operation of the robot (Figures 7-10; at least as in paragraphs 0054-0057, 0059-0061 and 0063-0072).
Regarding claim 7, in view of the above combination of Zhao and Yanagawa, Yanagawa further teaches the apparatus further comprising an input device, wherein the mode selection unit selects the operation mode from the plurality of operation modes based on an operation performed on the input device (Figures 3 & 4; at least as in paragraphs 0036-0039 and 0057-0060).
Regarding claim 8, in view of the above combination of Zhao and Yanagawa, Yanagawa further teaches wherein the input device is a touch panel, and
wherein a first switch and a second switch are displayed on the touch panel, the first switch used for selecting whether to operate the robot in the predetermined section under control based on the control signal and the second switch used for selecting whether to cause the imaging unit to acquire the reference image (Figures 3 & 4; at least as in paragraphs 0036-0039 and 0057-0060).
Regarding claim 9, in view of the above combination of Zhao and Yanagawa, Zhao further teaches a manufacturing method of an article, comprising manufacturing the article by using the robot apparatus according to claim 1 (at least as in paragraph 0020, and further as provided in the referenced sections of claim 1 above).
Regarding claim 10, Zhao teaches a method of controlling a robot apparatus including a robot (Figure 4, robot 400), including an imaging unit (Figure 4, camera 450) and a control unit (Figure 4, controller 440), configured to operate at a plurality of teaching points based on an instruction provided by the control unit (Figures 4-10; at least as in paragraphs 0037, 0041, 0072 and 0074, wherein “The robot 400 is in communication with a controller 440, which controls movement of the robot 400 and also receives data from the robot 400. A camera 450 is mounted on the outer robot arm such that the camera 450 is fixed in orientation relative to the gripper 402” and further wherein “At the box 1014, the trained pose estimation neural network is used in inference mode for visual servoing control of the robot. This could include any of the control system architectures illustrated in FIGS. 7-9, where the trained visual pose estimation neural network 530 is used in a pure visual servoing control system (FIG. 7), or the neural network 530 is used in visual servoing control for preliminary positioning followed by compliance control for final positioning (FIG. 8), or the neural network 530 is used in an integrated visual servoing/compliance control system (FIG. 9)”), the method comprising:
executing visual servoing based on a reference image that is an image representing a goal state of the robot and a captured image acquired by the imaging unit driven to each of the plurality of teaching points (Figures 4-10; at least as in paragraphs 0037, 0041, 0054, 0059, 0063 and 0072-0074, wherein “At the box 1014, the trained pose estimation neural network is used in inference mode for visual servoing control of the robot. This could include any of the control system architectures illustrated in FIGS. 7-9, where the trained visual pose estimation neural network 530 is used in a pure visual servoing control system (FIG. 7), or the neural network 530 is used in visual servoing control for preliminary positioning followed by compliance control for final positioning (FIG. 8), or the neural network 530 is used in an integrated visual servoing/compliance control system (FIG. 9)” and further wherein “the software applications and modules of these computers and controllers are executed on one or more computing devices having a processor and a memory module configured for learning visual pose estimation in robotic operations. In particular, this includes a processor in the robot controller 440 along with the optional separate computer used for data collection in FIG. 4, the computer used for the training process of FIG. 5, and the robot controllers and optional other computers which execute the functions of the visual servoing and compliance controllers”);
outputting a control signal for controlling the robot in a predetermined section (Figures 4-10; at least as in paragraphs 0037, 0041, 0054, 0059, 0063 and 0072-0074, wherein “At the box 1014, the trained pose estimation neural network is used in inference mode for visual servoing control of the robot. This could include any of the control system architectures illustrated in FIGS. 7-9, where the trained visual pose estimation neural network 530 is used in a pure visual servoing control system (FIG. 7), or the neural network 530 is used in visual servoing control for preliminary positioning followed by compliance control for final positioning (FIG. 8), or the neural network 530 is used in an integrated visual servoing/compliance control system (FIG. 9)” and further wherein “the software applications and modules of these computers and controllers are executed on one or more computing devices having a processor and a memory module configured for learning visual pose estimation in robotic operations. In particular, this includes a processor in the robot controller 440 along with the optional separate computer used for data collection in FIG. 4, the computer used for the training process of FIG. 5, and the robot controllers and optional other computers which execute the functions of the visual servoing and compliance controllers”); and
wherein, in a case a first operation mode is selected as the operation mode from the plurality of operation modes, the robot is moved to the teaching point and the imaging unit acquires the reference image (Figures 4-6 & 10; at least as in paragraphs 0036-0037 and 0039, wherein “the system of FIG. 4 is employed to collect data used for training a pose estimation neural network which, after training, will be used for visual servoing robot control. The data used for training is in the form of a plurality of images from the camera 450, with a robot pose recorded for each of the images. The robot pose which is recorded is the relative pose, in six degrees of freedom, of a tool center point with respect to a target pose”) Examiner notes wherein the offline data collection of images by the camera for training of a pose estimation (as detailed in at least Figure 4 of Zhao) is construed as a “first operation mode”., and
wherein, in a case where a second operation mode is selected as the operation mode from the plurality of operation modes, the robot is moved to the teaching point and the robot is controlled in the predetermined section based on the control signal (Figures 4-6 & 10; at least as in paragraphs 0041-0044, wherein “the images in the block 510 are taken from a variety of positions in the vicinity of the target pose, and under a variety of lighting conditions. The positions captured in the images may form a grid pattern or some other geometric or defined pattern surrounding the target pose, in which case the movement of the robot 400 and the triggering of the camera 450 to take each image may be defined in a control program” and further wherein “The block 520 contains the relative pose, recorded by the robot controller 440, corresponding with each image in the block 510. The relative pose defines the robot tool center point position relative to the target workpiece position, and was captured for each image”) Examiner notes wherein the learning of the pose estimation in a neural network (as detailed in at least Figures 5-6 of Zhao) is construed as a “second operation mode”.. That said, Zhao is silent specifically regarding wherein the robot apparatus specifically includes “selecting one of a plurality of operation modes as an operation mode of the robot at each of the plurality of teaching points”.
Yanagawa, in the same field of endeavor of teaching industrial robots/manipulators, teaches a programming method, apparatus and robot control system for teaching a robot to perform a task through teaching the robot to move and collect sensor data at different teaching points/positions associated with the task. Yanagawa further teaches wherein the robot may include a teach pendant that is connected to a robot controller of the robot, and further wherein said teach pendant may include a “mode selection key 32” for switching the robot between a plurality of operation modes (Figures 3 & 4; at least as in paragraphs 0036-0039 and 0057-0060, specifically as in at least paragraph 0057, wherein “when the mode selection key 32 in the teach pendant TP is operated by the operator, mode selection/execution process at S10 is performed. The mode selection/execution process allows execution of a teaching mode process at S20, sensing mode process at S30, or execution mode process at S40”). Therefore, it would have been obvious to one of ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Zhao to include Yanagawa’s teaching of providing the functionality of switching between a plurality of operating modes via a mode selection switch during programming (i.e. teaching) of said robot, since Yanagawa teaches wherein providing such functionality reduces the actual task cycle time by simplifying teaching tasks for the robot, thereby providing a more efficient and robust robot control system.
Regarding claim 11, in view of the above combination of Zhao and Yanagawa, Zhao further teaches a computer-readable recording medium storing a program for causing a computer to execute the method according to claim 10 (at least as in paragraph 0074, and further as provided in the referenced sections of claim 10 above).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892 – Notice of References Cited form. Examiner additionally notes the following prior art references, in the same field of endeavor as the instant invention, and also reads on several of the currently provided claim limitations above;
US 2017/0203434 A1, issued to Ueda, which is directed towards a robot and robot system for teaching a robot a plurality of teaching points with associated image data to perform one or more tasks.
US 2019/0308325 A1, issued to Higo, which is directed towards an image processing apparatus and corresponding method for a robot to perform one or more tasks, and further wherein said robot includes a mode switching unit for switching between an learning mode and an execution mode.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN L SAMPLE whose telephone number is (571)270-5925. The examiner can normally be reached Monday-Friday 7:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at (571)270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN L SAMPLE/Primary Examiner, Art Unit 3657
Prosecution Insights