DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-9 are pending.
Claims 10-20 have been withdrawn.
Response to Arguments
Applicant's amendments filed 04/14/2026 have been considered and entered, however, applicant’s arguments as filed have been fully considered but they are not persuasive.
Firstly, applicant failed to address the claim objections made previously, therefore those are repeated below.
Applicant further argues that cited references fail to teach the newly amended features of claim 1 such as “after calibrating, printing object with printhead assembly”. Applicant’s assertions seemed to be based on treating the rejection as a 102 instead of as 103 combination of Sugai in view of Lewis.
In reply, examiner disagrees and asserts that one cannot show non-obviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
In this case, Sugai teaches that workpiece W is recognized by using a sensor or a camera calibrated in base coordinate system, basically the information indicating the three-dimensional shape of the workpiece W is associated with base coordinate system and the image based on the print data is formed by N times of printing operation such that detection pattern is printed on the workpiece W or the object O while the robot 200 is driven by using the point data Dc indicating the ideal scanning path, paragraphs 68-70, wherein, printing apparatus 100 includes a robot 200, a liquid discharge head unit 300 as coupled together, see fig. 1, paragraphs 32-33 and imaging device 330 such as camera is coupled to liquid discharge head unit 300 that images the surface WF of the workpiece W, paragraphs 41, 69.
However, Lewis teaches robot is provided with a camera 160 and nozzles of print head for ejecting ink and “calibrating the camera based on the known position of the calibration artifact and the detected position of the calibration artifact; after calibrating, printing object with the printhead assembly” (based on the captured images of the markers, the robot can thus be controlled to position the print head more accurately with respect to the surface 11 to allow changes in position and orientation of the print head relative to the surface, paragraph 37, and as the robot moves relative to the surface 11 during printing, the fiducial markers remain in a line of sight of the camera, allowing a position and orientation of the print head relative to the camera to be estimated. Based on this estimate as well as on data provided by the internal position determination system, the controller 20 controls movement of the print head and activation of the nozzles of the print head during printing, paragraph 41).
Claim Objections
Claims 3-7 and 9 objected to because of the following informalities: There is semicolon is missing after the word comprising in claims 3-7 and 9. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over Sugai et al., US 2022/0097434 in view of Lewis et al., US 2020/0269602.
Regarding claim 1, Sugai discloses a method of calibrating a camera for use with robotic inkjet printing (camera is calibrated for using with inkjet printer 100 including a vertical articulated robot 200, paragraphs 32, 68) the method comprising:
printing a calibration artifact on an object (workpiece W), the calibration artifact comprises one or more control points (workpiece W is recognized by using a sensor or a camera calibrated in base coordinate system, basically the information indicating the three-dimensional shape of the workpiece W is associated with base coordinate system and the image based on the print data is formed by N times of printing operation such that detection pattern is printed on the workpiece W or the object O while the robot 200 is driven by using the point data Dc indicating the ideal scanning path, paragraphs 68-70), the calibration artifact (workpiece W) printed with a printhead assembly (liquid discharge head unit 300) that is coupled to a robot (printing apparatus 100 includes a robot 200, a liquid discharge head unit 300 as coupled together, see fig. 1) (see paragraphs 32-33) and the calibration artifact (workpiece W) is printed on an object at a known position (liquid discharge head unit 300 is a mechanism having a liquid discharge head 310 that discharges ink on the surface WF of the workpiece W by having fixed relationship between the positions and the poses, paragraph 41 and detection pattern is printed on the workpiece W while the robot 200 is driven by using the point data Dc indicating the ideal scanning path, and then the detection pattern is imaged by the imaging device 330, and the position on the scanning path is detected by using the imaging result, paragraph 69);
moving with the robot a camera that is coupled to the printhead assembly (imaging device 330 (imaging device 330 such as camera, paragraphs 44, 58) is coupled to liquid discharge head unit 300 as shown in fig. 1, paragraph 41) and capturing image data of the calibration artifact (imaging device 330 that images the surface WF of the workpiece W, paragraphs 41, 69);
detecting the calibration artifact from the image data (detection section 615 detects the position of the liquid discharge head 310 on the actual scanning path with respect to the workpiece W by using the imaging result of the imaging device 330, paragraph 69);
determining a detected position of the calibration artifact based on a location of the robot when the image data is captured (detection pattern is printed on the workpiece W while the robot 200 is driven by using the point data Dc indicating the ideal scanning path, and then the detection pattern is imaged by the imaging device 330, and the position on the scanning path is detected by using the imaging result, paragraphs 69, 130);
printing the object with the printhead assembly (liquid discharge head unit 300 is a mechanism having a liquid discharge head 310 that discharges ink on the surface WF of the workpiece W by having fixed relationship between the positions and the poses, paragraph 41 and detection pattern is printed on the workpiece W while the robot 200 is driven by using the point data Dc indicating the ideal scanning path, and then the detection pattern is imaged by the imaging device 330, and the position on the scanning path is detected by using the imaging result, paragraph 69).
Sugai fails to explicitly disclose calibrating camera based on known position of calibration artifact and detected position of the calibration artifact; after calibrating, printing object with printhead.
However, Lewis teaches moving with robot a camera that is coupled to printhead assembly and capturing image data of calibration artifact (robot is provided with a camera 160 and nozzles of print head for ejecting ink. As the print head is moved across the surface 11 while the nozzles are controlled to print ink on the surface, the camera 160 captures images of the surface, paragraphs 36-38, 18); and calibrating the camera based on the known position of the calibration artifact and the detected position of the calibration artifact; after calibrating, printing object with the printhead assembly (based on the captured images of the markers, the robot can thus be controlled to position the print head more accurately with respect to the surface 11 to allow changes in position and orientation of the print head relative to the surface, paragraph 37, and as the robot moves relative to the surface 11 during printing, the fiducial markers remain in a line of sight of the camera, allowing a position and orientation of the print head relative to the camera to be estimated. Based on this estimate as well as on data provided by the internal position determination system, the controller 20 controls movement of the print head and activation of the nozzles of the print head during printing, paragraph 41).
Sugai and Lewis are combinable because they both are in the same field of endeavor dealing with printhead assembly mounted on a robot.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai with the teachings of Lewis for the benefit of being able to effectively stabilize the print head using high speed actuators connecting the print head to the robotic arm as taught by Lewis at paragraph 27.
Regarding claim 3, Sugai further discloses creating scan paths for the robot to move the printhead assembly over the object and capturing the image data of the calibration artifact while the robot is moving the printhead assembly along the scan paths (detecting the position of the liquid discharge head 310 on the actual scanning path with respect to the workpiece W by using the imaging result of the imaging device 330, wherein, a detection pattern is printed on the workpiece W while the robot 200 is driven by using the point data Dc indicating the above-mentioned ideal scanning path, and then the detection pattern and the position on the scanning path is detected by using the imaging result, paragraph 69 and, in step S132, the detection pattern is printed on the workpiece W while the robot 200 is operated by using the point data Dc generated in step S131. Then, in step S133, the actual scanning path in step S132 is detected, paragraph 98).
Claims 2 and 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Sugai et al., US 2022/0097434 in view of Lewis et al., US 2020/0269602 as applied in claim 1 above and further in view of Ishige et al., US 2017/0151671.
Regarding claim 2, Sugai further discloses wherein the calibration artifact comprises one or more control points that are located at known control point positions (point data Dc is generated indicating the scanning path of the liquid discharge head 310 with respect to the workpiece W with the coordinate values of the base coordinate system, paragraph 59), the method further comprising:
determining detected positions of the control points from the image data captured by the camera (detection section detects the position of the liquid discharge head 310 on the actual scanning path with respect to the workpiece W by using the imaging result of the imaging device and also the detection pattern is imaged by the imaging device 330, and the position on the scanning path is detected by using the imaging result, paragraph 69); and calibrating the camera based on the positions of the control points of the calibration artifact (correcting the point data Dc indicating the ideal scanning path based on the detection result of the detection section in the generation of the workpiece information Da, where, the workpiece W is recognized by using a sensor or a camera calibrated in the base coordinate system, and the information indicating the three-dimensional shape of the workpiece W is associated with the base coordinate system, paragraph 68).
Combination of Sugai with Lewis fails to further teach calibrating camera based on positions of control points of target/object and detected positions of the control points of the target/object.
However, Ishige teaches calibrating camera based on positions of control points of target/object and detected positions of the control points of the target/object (acquiring the position and the posture in the image coordinate system on the image of the fixed camera 2 into the coordinate in the robot coordinate system, that is, the calibration of the fixed camera 2. Similarly, in order to allow the robot 1 to accurately perform the work with respect to the target or the like based on the image captured by the mobile camera 3, it is necessary to perform processing of acquiring the correction parameter for converting the position and the posture in the image coordinate system on the image of the mobile camera 3 into the coordinate in the robot coordinate system, that is, the calibration of the mobile camera 3, paragraph 101).
Sugai and Lewis are combinable with Ishige because they all are in the same field of endeavor dealing with mechanical movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Ishige for the benefit of effectively determining the position and the posture of the calibration tool in order for the determination of the contact state to be accurately performed by a worker as taught by Ishige at paragraphs 9-10.
Regarding claim 6, Sugai further discloses determining the location of the robot based on a tool coordinate system of the printhead assembly (a-axis, the b-axis, and the c-axis are coordinate axes of a tool coordinate system set in the liquid discharge head unit 300, and a relationship between a position and a pose relative to the above-mentioned X-axis, Y-axis, and Z-axis changes by the operation of the above-mentioned robot 200, paragraph 73).
Combination of Sugai with Lewis fails to further teach determining position/location of robot based on a tool center point of assembly.
However, Ishige teaches determining position/location of robot based on a tool center point of assembly (center of the tip end surface of the robot hand of robot’s arm is referred to as a tool center point and determining the reference surface may be determined based on the coordinate of the tool center point TCP such that the position and the posture of the second marker may be instructed to the robot, paragraphs 70, 74, 133, 203).
Sugai and Lewis are combinable with Ishige because they all are in the same field of endeavor dealing with mechanical movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Ishige for the benefit of effectively determining the position and the posture of the calibration tool in order for the determination of the contact state to be accurately performed by a worker as taught by Ishige at paragraphs 9-10.
Regarding claim 7, Sugai further discloses calibrating based on distance on printhead assembly between tool coordinate system (a-axis, the b-axis, and the c-axis are coordinate axes of a tool coordinate system set in the liquid discharge head unit 300, and a relationship between a position and a pose relative to the above-mentioned X-axis, Y-axis, and Z-axis changes by the operation of the above-mentioned robot 200, paragraph 73).
Combination of Sugai with Lewis fails to further teach calibrating camera based on a known distance on robot assembly between tool center point and the camera.
However, Ishige teaches calibrating camera based on a known distance on robot assembly between tool center point and the camera (posture of the reference surface based on the images captured by the fixed camera, it is possible to acquire the posture of the reference surface with high accuracy and by bringing an instruction tool (touch-up hand) of which the offset is known in the axial coordinates into contact with the second marker, which was captured by the image of the second marker by the camera, it is preferable to instruct the position and the posture of the second marker to the robot, wherein, when the distance between the axial coordinates and a tool center point TCP is known in advance, it is possible to acquire the surface including the tool center point TCP based on the distance and the reference surface which is the surface including the axial coordinates, thereby, it is possible to perform calibration of camera by acquiring the posture of the reference surface based on the distances, paragraphs 128, 132, 135, 203).
Sugai and Lewis are combinable with Ishige because they all are in the same field of endeavor dealing with mechanical movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Ishige for the benefit of effectively determining the position and the posture of the calibration tool in order for the determination of the contact state to be accurately performed by a worker as taught by Ishige at paragraphs 9-10.
Regarding claim 8, Sugai further discloses wherein the calibration artifact is a first calibration artifact (Sugai, printing on the surface of the three-dimensional workpiece W by an ink jet method, paragraphs 29-30), the method further comprising: printing additional calibration artifacts on the object at known positions (Sugai, an object O corresponding to the workpiece W is used if necessary. The object O is an object having a surface OF having substantially the same shape and pose as the surface WF. The object O may be a peelable film attached to the surface WF of the workpiece W, paragraph 31);
capturing additional image data of the additional calibration artifacts (Sugai, detecting the position of the liquid discharge head 310 on the actual scanning path with respect to the object O by using the imaging result of the imaging device 330, wherein, a detection pattern is printed on the object O while the robot 200 is driven by using the point data Dc indicating the above-mentioned ideal scanning path, and then the detection pattern and the position on the scanning path is detected by using the imaging result, paragraph 69);
and calibrating the camera with respect to the robot based on a difference between the known positions of the calibration artifacts when the image data of the calibration artifacts was captured (generates the workpiece information Da, generates the point data Dc indicating an ideal scanning path based on the workpiece information Da, and then corrects the point data Dc indicating the ideal scanning path based on the detection result of the detection section, wherein, generation of the workpiece or Object O information Da, workpiece or Object O is recognized by using a sensor or a camera calibrated in the base coordinate system, and the information indicating the three-dimensional shape of the workpiece or Object O is associated with the base coordinate system, paragraph 68).
Combination of Sugai with Lewis fails to further teach calibrating camera with respect to robot based on a difference between positions of target and tool center point when image data was captured.
However, Ishige teaches calibrating camera with respect to robot based on a difference between positions of target and tool center point when image data was captured (in order to allow the robot 1 to accurately perform the work with respect to the target or the like based on the image captured by the mobile camera 3, it is necessary to perform processing of acquiring the correction parameter for converting the position and the posture in the image coordinate system on the image of the mobile camera 3 into the coordinate in the robot coordinate system, that is, the calibration of the mobile camera 3, paragraph 101 and it is preferable to instruct the position and the posture of the second marker to the robot, wherein, when the distance between the axial coordinates and a tool center point TCP is known in advance, it is possible to acquire the surface including the tool center point TCP based on the positions of the reference surface which is the surface including the axial coordinates, paragraphs 132, 203).
Sugai and Lewis are combinable with Ishige because they all are in the same field of endeavor dealing with mechanical movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Ishige for the benefit of effectively determining the position and the posture of the calibration tool in order for the determination of the contact state to be accurately performed by a worker as taught by Ishige at paragraphs 9-10.
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Sugai et al., US 2022/0097434 in view of Lewis et al., US 2020/0269602 as applied in claim 1 above and further in view of Inanoglu et al., US 2025/0209588.
Regarding claim 4, Sugai further discloses capturing three-dimensional scans of a surface of the object prior to printing the calibration artifact at the known position (workpiece W is recognized by using a camera calibrated in the base coordinate system, and the information indicating the three-dimensional shape of the workpiece W is associated with the base coordinate system and detection section detects the actual position of the scanning path of the liquid discharge head, prior to a printing operation, which prints an image based on the print data, paragraphs 68-69).
Combination of Sugai with Lewis fails to further teach performing three-dimensional scans of a surface of object with a metrology system.
However, Inanoglu teaches performing three-dimensional scans of a surface of object with a metrology system (measuring instrument is mounted to a robot (robotic arm) that moves the three-dimensional (3D) metrology system along a trajectory to perform a 3D scan to obtain 3D measurements of a surface of an object, paragraph 6).
Sugai and Lewis are combinable with Inanoglu because they all are in the same field of endeavor dealing with movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Inanoglu for the benefit of being able to effectively displacing the robot along the scanning trajectory between the trajectory start point and the trajectory end point to obtain accurate 3D measurements of the surface of the object as taught by Inanoglu at paragraph 39.
Regarding claim 5, Combination of Sugai with Lewis further teaches determining the movement of the robot in a coordinate system that is based on the three-dimensional scans of the object (workpiece W is recognized by using a camera calibrated in the base coordinate system, and the information indicating the three-dimensional shape of the workpiece W is associated with the base coordinate system, paragraphs 68, 29 and Lewis teaches that as the robot moves relative to the surface 11, the fiducial markers remain in a line of sight of the camera, allowing a three-dimensional (3D, paragraph 21) position and orientation of the print head relative to the camera to be estimated, paragraph 41).
Sugai and Lewis are combinable because they both are in the same field of endeavor dealing with printhead assembly mounted on a robot.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai with the teachings of Lewis for the benefit of being able to effectively stabilize the print head using high speed actuators connecting the print head to the robotic arm as taught by Lewis at paragraph 27.
Combination of Sugai with Lewis fails to further teach measure/moving of robot in a metrology coordinate system that is based on three-dimensional scans of object.
However, Inanoglu teaches measure/moving of robot in a metrology coordinate system that is based on three-dimensional scans of object (measuring instrument is mounted to a robot (robotic arm) that moves the three-dimensional (3D) metrology system along a trajectory to perform a 3D scan to obtain 3D measurements of a surface of an object, paragraph 6).
Sugai and Lewis are combinable with Inanoglu because they all are in the same field of endeavor dealing with movement of robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Inanoglu for the benefit of being able to effectively displacing the robot along the scanning trajectory between the trajectory start point and the trajectory end point to obtain accurate 3D measurements of the surface of the object as taught by Inanoglu at paragraph 39.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sugai et al., US 2022/0097434 in view of Lewis et al., US 2020/0269602 as applied in claim 1 above and further in view of Steger et al., US 2021/0327136.
Regarding claim 9, Sugai further discloses wherein capturing the image data of the calibration artifact comprises capturing the image data with a camera (imaging device 330 can be a camera and also includes an imaging element comprising CMOS image sensor, paragraphs 68, 44).
Combination of Sugai with Lewis fails to further teach wherein capturing image data comprises capturing the image data with a line scan camera.
However, Steger teaches wherein capturing image data comprises capturing the image data with a line scan camera (image data can be captured by a line scan camera mounted on a robotic arm, paragraph 54).
Sugai and Lewis are combinable with Steger because they all are in the same field of endeavor dealing with camera mounted on robotic arms.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to combine the teachings of Sugai and Lewis with the teachings of Steger for the benefit of effectively and accurately reconstructing a 3D scene with line-scan cameras having telecentric lenses as taught by Steger at paragraph 16.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tenney et al., US 2016/0035079
Parsons, US 2023/0153978
Smith et al., US 11,407,166
Yaghoobi et al., US 2021/0331193
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAWANDEEP DHINGRA whose telephone number is (571) 270-1231. The examiner can normally be reached 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at (571) 270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAWAN DHINGRA/Examiner, Art Unit 2683
/ABDERRAHIM MEROUAN/Supervisory Patent Examiner, Art Unit 2683
Prosecution Insights