DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-4 and 6-10 are pending.
Claim 5 is canceled.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-4 and 6-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yung et al (US20190184570) in view of Nakayama (US20170165803).
Regarding claims 1, 6 and 7, Yung teaches a specific point detection system comprising:
(Yung, "One embodiment can provide an intelligent robotic system.", [0005]; a detection system by providing an intelligent robotic system capable of identifying and locating specific components)
a robot arm;
(Yung, Fig. 1; "at least one multi-axis robotic arm", [0005]; "Intelligent robot 100 comprises a base 102 and a multi-joint arm 104", [0034])
an imager comprising a camera that acquires an image of an object;
(Yung, Fig. 2; " wide-field-of-view camera system 202 can be attached to a lower portion of the robot arm (i.e., arm section 204), close to the base of the robot, overseeing work surface 210. Because arm section 204 is coupled to the base of the robot via a rotational joint 206, as arm section 204 rotates, wide-field-of-view camera system 206 can scan a large range of work surface 210 to search for component 212", [0038]; an imager with a camera that captures wide-field images of the work surface and objects/components on it)
a first detector that detects, using a first detection model, a specific point included in the image of the object, the specific point comprising a deformation to be removed from a surface of the object, the first detector detecting the specific point by taking the image acquired by the imager as input, the first detection model outputting a position of the specific point on the surface of the object;
(Yung, Fig. 2; "In some embodiments, a pre-trained neural network can be used to identify the desired component.", [0043]; "In the first step, based on wide-angle images of the work surface and a CNN trained for classifying components, the vision system of the robot can identify and locate a desired component on the work surface.", [0062]; "In some embodiments, wide-field-of-view camera system 202 may also acquire the pose (including position and orientation) of component 212 in order to allow for more accurate motion planning.", [0039]; Yung teaches a first detector using a first detection model (CNN) taking the wide-angle image as input to detect the specific point/component and output its position. Nakayama, Figs. 1, 3, 5A; "removal tools for removing machining chips", [0045]; "The robot 3 cuts off the machining chips C away from the workpiece W by moving the clipping tool 43 along the surface of the workpiece W.", [0047]; "an image processing unit which detects the position and amount of the machining chips adhering to the workpiece", [0008]; Nakayama teaches detecting a specific point comprising a deformation (machining chips) to be removed from a surface of the object. Yung does not explicitly disclose the detected specific point as comprising a deformation to be removed from a surface of the object. However, Nakayama teaches an image processing unit detecting and outputting the position of deformations (machining chips) adhering to the surface of a workpiece so a robot can remove them)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the teachings of Nakayama into the system or method of Yung in order to enable the intelligent robot's vision system to detect and remove unwanted deformations such as machining chips from the work surface. The combination of Yung and Nakayama also teaches other enhanced capabilities.
The combination of Yung and Nakayama further teaches:
a three-dimensional information acquirer comprising a three-dimensional scanner that acquires, independently from the camera and the acquired image, three-dimensional information on the object including the specific point detected by the first detector; and
(Yung, Fig. 2; "a 3D surface-imaging system, which can include a close-range camera and a structured-light projector", [0043]; "Close-range camera 222 can be attached to the stem or base of the gripper ... structured-light projector 226 can also be installed in the vicinity of close-range camera 222", [0040]; "A fine 3D point cloud can then be generated based on the captured images.", [0048]; "The close range can provide high resolution images as well as a high-accuracy computation of depth information.", [0041]; a 3D information acquirer (close-range camera and projector) that operates independently of the wide-field camera to acquire high-resolution 3D information/point cloud of the detected object)
a second detector that, based on the position of the specific point output by the first detection model, re-detects, using a second detection model, the specific point by taking the three-dimensional information acquired by the three-dimensional information acquirer as input, wherein
(Yung, " Once the component is identified and located, a pose-classifying CNN that is specific to the identified component can be used to recognize the pose and location in higher resolution and confidence. Note that inputs to the pose-classifying CNN can include high- resolution 2D or 3D images.", [0062]; this is a second detector using a second detection model (pose-classifying CNN) that takes the 3D images as input to re-detect the location and pose of the specific point in higher resolution)
the imager and the three-dimensional information acquirer are located on the robot arm, and
(Yung, Fig. 2; "wide-field-of-view camera system 202 can be attached to a lower portion of the robot arm", [0038]; "Close-range camera 222 can be attached to the stem or base of the gripper, which is attached to wrist joint 224 of the robot arm.", [0040]; both the imager (wide-field) and 3D information acquirer (close-range) are physically located on the robot arm)
the robot arm performs operations comprising:
moving the imager to a predetermined imaging position to acquire the image of an entirety of the object, and
(Yung, "control movements of the multi-axis robotic arm", [0005]; "as arm section 204 rotates, wide-field-of-view camera system 206 can scan a large range of work surface 210 to search for component 212", [0038]; "the robot first searches the work surface to locate a particular component", [0043]; moving the imager via arm rotation to scan and acquire a wide-angle image of the entirety of the work surface/object)
moving the three-dimensional information acquirer to a position based on the position of the specific point detected and output during the first detection by the imager, to acquire the three-dimensional information on an entirety of the object and including the specific point detected by the first detector.
(Yung, "Subsequent to locating the desired component on the work surface, the robot can move its gripper along with a 3D surface-imaging system, which can include a close-range camera and a structured-light projector, toward the vicinity of the component", [0043]; "The structured-light projector can project a dotted pattern onto the part of the work surface that includes the desired component", [0049]; moving the 3D surface-imaging system to a position based on the initial location output by the wide-field imager in order to acquire 3D information covering the component/specific point)
Regarding claim 2, the combination of Yung and Nakayama teaches its/their respective base claim(s).
The combination further teaches the specific point detection system of claim 1, wherein an area of the object input to the second detection model as the three-dimensional information is narrower than an area of the object input to the first detection model as the image.
(Yung, see comments on claim 1; “3D point cloud can be generated, which includes 3D data of objects within the field of view of close-range camera 222”, [0041]; camera 222/ structured light projector 226 (“a second detector”) is a close-range 3D imaging system with high 3D accuracy and has a narrower field of view than the wide-field-of-view camera system 202 (“first detector”))
Regarding claim 3, the combination of Yung and Nakayama teaches its/their respective base claim(s).
The combination further teaches the specific point detection system of claim 1, wherein the three-dimensional information is point cloud data.
(Yung, Fig. 3, “the robot can generate a 3D point cloud describing the area surrounding the component and recommend a pick up position for the gripper (operation 312)”, [0043])
Regarding claim 4, the combination of Yung and Nakayama teaches its/their respective base claim(s).
The combination further teaches the specific point detection system of claim 1, wherein
(Yung, see comments on claim 1 and note the 112b rejection to the claim; “the robot can turn on structured-light projector 226 in response to ultrasonic sensor 228 detecting that the distance to component 212 is less than a predetermined value (e.g., 150 mm) ... The close range can provide high resolution images as well as a high-accuracy computation of depth information”, [0041]; camera 222 provides higher accuracy of depth measurement with structured light projector 226 (“second detection model”) meaning higher explainability than the rough depth measurement system of wide-field-of-view camera system 202 (camera 214 with structured light projector 216 => “first detection model”))
Regarding claims 8, 9 and 10, the combination of Yung and Nakayama teaches its/their respective base claim(s).
The combination further teaches the specific point detection system according to claim 1, further comprising:
a robot arm; and
(Yung, see comments on claim 5)
a controller, wherein the controller is configured to control the robot arm to automatically remove the deformation at the detected position of the specific point.
(Nakayama, “if there are any machining chips adhering to the specific portion, the step of removing such machining chips from the machining tool is automatically executed”, [0107])
Response to Arguments
Applicant's arguments filed on 3/23/2026 with respect to one or more of the pending claims have been fully considered but they are not persuasive.
Regarding claim(s) 1, 6 and 7, Applicant, in the remarks, argues that the combination of the cited reference(s) fails to teach the newly amended limitations in the claims.
The Examiner respectfully disagreed. The office action has been updated to address applicant’s argument. See the updated review comments for details.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 5/2/2026
Prosecution Insights