Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This communication is responsive to Application No. 18/794,667 and the preliminary
amendments filed on 8/9/2024.
3. Claims 1-21 are presented for examination.
Information Disclosure Statement
4. The information disclosure statements (IDS) submitted on 10/29/2024, 11/22/2024, 2/24/2025, 4/25/2025, 8/25/2025, and 10/6/2025 have been considered by the examiner.
5. The Applicant has submitted six information disclosure statements (IDSs), comprising 89 pre-grant publications, 67 patents, 30 foreign patent documents, and 49 non-patent literature (NPL) references, totaling 235 documents, which is excessive for any Examiner to have to consider at any level beyond a cursory review. In accord with dicta from Molins PLC v. Textron, Inc., 48 F.3d 1172 (Fed. Cir. 1995), stating that forcing the Examiner to find "a needle
in a haystack" is "probative of bad faith." Id. [The Molins] case presented a situation where the
disclosure was in excess of 700 pages and contained more than fifty references. Likewise, the
instant application’s IDSs include more references than even the Molins case, and these
particularly long IDSs do not include any concise explanation of the relevance of any of the
listed references nor cite any pages, columns, and lines (or paragraph numbers) where relevant
passages or relevant figures appear. According to MPEP Section 2004 “Aids to Compliance With
Duty of Disclosure [R-08.2012]”, “It is desirable to avoid the submission of long lists of documents if it can be avoided. Eliminate clearly irrelevant and marginally pertinent cumulative
information. If a long list is submitted, highlight those documents which have been specifically
brought to Applicant’s attention and/or are known to be of most significance.” Additionally, per
MPEP Section 609.04(a)(III): “applicants are encouraged to provide a concise explanation of why the English-language information is being submitted and how it is understood to be relevant. Concise explanations (especially those which point out the relevant pages and lines) are helpful to the Office, particularly where documents are lengthy and complex and applicant is aware of a section that is highly relevant to patentability or where a large number of documents are submitted and applicant is aware that one or more are highly relevant to patentability.” See Penn Yan Boats, Inc. v. Sea Lark Boats, Inc., 359 F. Supp. 948, 175 USPQ 260 (S.D. Fla. 1972), aff’d, 479 F.2d 1338, 178 USPQ 577 (5th Cir. 1973), cert. denied, 414 U.S. 874 (1974). But cf. Molins PLC v.Textron Inc., 48 F.3d 1172, 33 USPQ2d 1823 (Fed. Cir. 1995). As such, even though these IDSs have been placed in the application file with the lists of references marked as considered, and the compilation of those listed PG Publications and Patent references have at least been key-word searched and/or classification searched for relevant prior art, the information referred to therein for each individual reference has admittedly not been fully considered beyond a cursory review. If Applicant wishes to have one or more references fully considered, the Examiner requests resubmitting the IDSs with a reasonable number of references that are known to be pertinent for the determination of patentability as defined by 37 C.F.R. § 1.56, along with the concise explanations as to relevance and citations explaining the locations of relevant passages or figures, as per 37 CFR 1.98(a)(3) and 37 CFR § 1.105.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claim(s) 1, 8, 11, 18, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov).
Regarding Claim 1, Oota teaches a computer-implemented method for following at least a portion of an edge of an object via an edge-following system, the computer-implemented method comprising an edge-following operation, wherein the edge-following operation comprises at least: causing a handling tool associated with a multi-axis robot to engage the object ([0035] via “As shown in FIG. 2, the robot 210 includes a robot hand 201, the posture of which is controlled to various positions and angles. The robot 200, for example, grips in series the workpieces 50 being a plurality of objects to be inspected that are prepared in a workpiece storage space. The robot hand 201 can change the position and posture of the gripped workpiece 50.”);
defining a working point on the edge of the object ([0037] via “The control device 300
causes the camera 210 to image in each imaging point set on the surface to be inspected of the workpiece 50, while moving the robot hand 201 gripping the workpiece 50, along a movement route including a plurality of imaging points set on the surface to be inspected so that the surface to be inspected of the workpiece 50 is entirely covered by a plurality of images imaged by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota wherein the imaging points (interpreted to be a working point) are located on the edges of the workpiece.), wherein the working point is kept at a predetermined working offset from an ancillary tool associated with the edge-following system ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”); and
causing movement of the multi-axis robot to manipulate the object via the handling tool
while maintaining the predetermined working offset continuously between the ancillary tool and the working point ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … Thus, the accuracy of the flaw inspection can be improved by, in addition to the imaging in which the imaging region including the imaging point is perpendicular to the optical axis of the camera 210 (and the illumination light of the illumination 220), for example, as shown in FIG. 4C, adjusting the orientation of the workpiece 50 gripped by the robot hand 201, by the operation of the robot hand 201, for example, as shown in FIG. 4D or FIG. 4E, so that, in the same imaging point, the imaging region including the imaging point has an angle that is not perpendicular to the optical axis of the camera 210 and the illumination light of the illumination 220. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50
gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”), ([0041] via “The movement operation control unit 330 moves the robot hand 201 on the basis of the movement route of the robot hand 201 calculated by the imaging position information setting unit 310. Thereby, the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is controlled so that all imaging points included in the imaging position information set by the imaging position information setting unit 310 are covered by the imaging points where imaging is performed by the camera 210.”).
Oota is silent on determining one or more dimensional attributes associated with the object; causing movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side.
However, Diankov teaches determining one or more dimensional attributes associated with the object ([0041] via “In some embodiments, the destination crossing sensor 316 can be used to measure a height of the target object 112 during transfer. … The robotic system 100
can compare the gripper height 322 to a crossing reference height 324 (e.g., a known vertical position of the destination crossing sensor 316 and/or a reference line/plane thereof) to calculate an object height 320 of the target object 112 that is being transferred.”);
causing movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330. Also, the robotic system 100 can derive the scanning maneuver 404 for rotating the target object 112 to present multiple surfaces thereof to the scanning sensor 330.”), ([0056] via “Also, as an illustrative example, the robotic system 100 can derive scanning maneuver 414 for moving the recognized object along the x-axis and/or the y-axis and/or for rotating the object about the z-axis.”), (Note: See Figures 4A-4D of Diankov as well.);
wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side ([0024] via “In some embodiments, the task can include manipulation (e.g., moving and/or reorienting) of a target object 112 (e.g., one of the packages, boxes, cases, cages, pallets, etc. corresponding to the executing task) from a start location 114 to a task location 116.”), ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330.”), (Note: See Figures 4A-4D and 6A-6F of Diankov as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diankov wherein the edge-following operation comprises at least: determining one or more dimensional attributes associated with the object; causing movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side. Doing so allows the system to accurately recognize all dimensions and surfaces of the object for further object processing, as stated by Diankov ([0054] via “The robotic system 100 can operate the scanning sensor 330 based on placing the end effector 304 at the scanning position 412 and/or based on implementing the scanning maneuver 414. … Thus, based on the scanning position 412 and/or the scanning maneuver 414, the robotic system 100 can present multiple surfaces/portions of the unrecognized objects and increase the likelihood of accurately locating and scanning identifiers on the unrecognized objects.”).
Regarding Claim 8, modified reference Oota teaches the computer-implemented method of claim 1, but is silent on wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object.
However, Diankov teaches wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330. Also, the robotic system 100 can derive the scanning maneuver 404 for rotating the target object 112
to present multiple surfaces thereof to the scanning sensor 330.”), ([0056] via “Also, as an illustrative example, the robotic system 100 can derive scanning maneuver 414 for moving the recognized object along the x-axis and/or the y-axis and/or for rotating the object about the z-axis.”), (Note: See Figures 4A-4D of Diankov as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diankov wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object. Doing so allows the system to accurately recognize all dimensions and surfaces of the object for further object processing, as stated by Diankov ([0054] via “The robotic system 100 can operate the scanning sensor 330 based on placing the end effector
304 at the scanning position 412 and/or based on implementing the scanning maneuver 414. … Thus, based on the scanning position 412 and/or the scanning maneuver 414, the robotic system 100 can present multiple surfaces/portions of the unrecognized objects and increase the likelihood of accurately locating and scanning identifiers on the unrecognized objects.”).
Regarding Claim 11, Oota teaches an edge-following system for following an edge of an object, the edge-following system comprising at least one processor and at least one non-transitory memory including computer-coded instructions thereon, the computer-coded instructions, when executed by the at least one processor ([0077] via “The function blocks included in the machine learning device 10, the control device 300, and the flaw inspection device 400 are described above. In order to realize these function blocks, the machine learning device 10, the control device 300, and the flaw inspection device 400 include an operation processing device such as a central processing unit (CPU). The machine learning device 10, the control device 300, and the flaw inspection device 400 also include a sub storage device such as a hard disk drive (HDD) stored with various control programs such as application software and an operating system (OS), and a main storage device such as a random access memory (RAM) for storing data temporarily required for execution of the program by the operation processing device.”), ([0096] via “The program may be stored by using various types of non-transitory computer readable media, and supplied to the computer.”), cause the edge-following system to:
cause a handling tool associated with a multi-axis robot to engage the object ([0035] via “As shown in FIG. 2, the robot 210 includes a robot hand 201, the posture of which is controlled to various positions and angles. The robot 200, for example, grips in series the workpieces 50 being a plurality of objects to be inspected that are prepared in a workpiece storage space. The robot hand 201 can change the position and posture of the gripped workpiece 50.”);
define a working point on the edge of the object ([0037] via “The control device 300
causes the camera 210 to image in each imaging point set on the surface to be inspected of the workpiece 50, while moving the robot hand 201 gripping the workpiece 50, along a movement route including a plurality of imaging points set on the surface to be inspected so that the surface to be inspected of the workpiece 50 is entirely covered by a plurality of images imaged by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota wherein the imaging points (interpreted to be a working point) are located on the edges of the workpiece.), wherein the working point is kept at a predetermined working offset from an ancillary tool associated with the edge-following system ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”); and
cause movement of the multi-axis robot to manipulate the object via the handling tool while maintaining the predetermined working offset continuously between the ancillary tool and the working point ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … Thus, the accuracy of the flaw inspection can be improved by, in addition to the imaging in which the imaging region including the imaging point is perpendicular to the optical axis of the camera 210 (and the illumination light of the illumination 220), for example, as shown in FIG. 4C, adjusting the orientation of the workpiece 50 gripped by the robot hand 201, by the operation of the robot hand 201, for example, as shown in FIG. 4D or FIG. 4E, so that, in the same imaging point, the imaging region including the imaging point has an angle that is not perpendicular to the optical axis of the camera 210 and the illumination light of the illumination 220. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50
gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”), ([0041] via “The movement operation control unit 330 moves the robot hand 201 on the basis of the movement route of the robot hand 201 calculated by the imaging position information setting unit 310. Thereby, the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is controlled so that all imaging points included in the imaging position information set by the imaging position information setting unit 310 are covered by the imaging points where imaging is performed by the camera 210.”).
Oota is silent on to determine one or more dimensional attributes associated with the object; cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side.
However, Diankov teaches to determine one or more dimensional attributes associated with the object ([0041] via “In some embodiments, the destination crossing sensor 316 can be used to measure a height of the target object 112 during transfer. … The robotic system 100
can compare the gripper height 322 to a crossing reference height 324 (e.g., a known vertical position of the destination crossing sensor 316 and/or a reference line/plane thereof) to calculate an object height 320 of the target object 112 that is being transferred.”);
cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330. Also, the robotic system 100 can derive the scanning maneuver 404 for rotating the target object 112 to present multiple surfaces thereof to the scanning sensor 330.”), ([0056] via “Also, as an illustrative example, the robotic system
100 can derive scanning maneuver 414 for moving the recognized object along the x-axis and/or the y-axis and/or for rotating the object about the z-axis.”), (Note: See Figures 4A-4D of Diankov as well.);
wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side ([0024] via “In some embodiments, the task can include manipulation (e.g., moving and/or reorienting) of a target object 112 (e.g., one of the packages, boxes, cases, cages, pallets, etc. corresponding to the executing task) from a start location 114 to a task location 116.”), ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330.”), (Note: See Figures 4A-4D and 6A-6F of Diankov as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diankov wherein the edge-following system is caused to: determine one or more dimensional attributes associated with the object; cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side. Doing so allows the system to accurately recognize all dimensions and surfaces of the object for further object processing, as stated by Diankov ([0054] via “The robotic system 100
can operate the scanning sensor 330 based on placing the end effector 304 at the scanning position 412 and/or based on implementing the scanning maneuver 414. … Thus, based on the scanning position 412 and/or the scanning maneuver 414, the robotic system 100 can present multiple surfaces/portions of the unrecognized objects and increase the likelihood of accurately locating and scanning identifiers on the unrecognized objects.”).
Regarding Claim 18, modified reference Oota teaches the edge-following system of claim 11, but is silent on wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object.
However, Diankov teaches wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330. Also, the robotic system 100 can derive the scanning maneuver 404 for rotating the target object 112
to present multiple surfaces thereof to the scanning sensor 330.”), ([0056] via “Also, as an illustrative example, the robotic system 100 can derive scanning maneuver 414 for moving the recognized object along the x-axis and/or the y-axis and/or for rotating the object about the z-axis.”), (Note: See Figures 4A-4D of Diankov as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diankov wherein causing the movement of the multi-axis robot to manipulate the object comprises executing a sequence of alternately translating the object linearly along one or more of an x-axis or a y-axis of a particular coordinate plane and rotating the object based in part on one or more points of rotation to capture image data associated with the plurality of sides and the corners comprised in the edge of the object. Doing so allows the system to accurately recognize all dimensions and surfaces of the object for further object processing, as stated by Diankov ([0054] via “The robotic system 100 can operate the scanning sensor 330 based on placing the end effector
304 at the scanning position 412 and/or based on implementing the scanning maneuver 414. … Thus, based on the scanning position 412 and/or the scanning maneuver 414, the robotic system 100 can present multiple surfaces/portions of the unrecognized objects and increase the likelihood of accurately locating and scanning identifiers on the unrecognized objects.”).
Regarding Claim 21, Oota teaches at least one non-transitory computer-readable storage medium for following an edge of an object, the at least one non-transitory computer-readable storage medium having computer program code stored thereon that, in execution with at least one processor, configures the at least one processor ([0077] via “The function blocks included in the machine learning device 10, the control device 300, and the flaw inspection device 400 are described above. In order to realize these function blocks, the machine learning device 10, the control device 300, and the flaw inspection device 400
include an operation processing device such as a central processing unit (CPU). The machine learning device 10, the control device 300, and the flaw inspection device 400 also include a sub storage device such as a hard disk drive (HDD) stored with various control programs such as application software and an operating system (OS), and a main storage device such as a random access memory (RAM) for storing data temporarily required for execution of the program by the operation processing device.”), ([0096] via “The program may be stored by using various types of non-transitory computer readable media, and supplied to the computer.”) to:
cause a handling tool associated with a multi-axis robot to engage the object ([0035] via “As shown in FIG. 2, the robot 210 includes a robot hand 201, the posture of which is controlled to various positions and angles. The robot 200, for example, grips in series the workpieces 50 being a plurality of objects to be inspected that are prepared in a workpiece storage space. The robot hand 201 can change the position and posture of the gripped workpiece 50.”);
define a working point on the edge of the object ([0037] via “The control device 300
causes the camera 210 to image in each imaging point set on the surface to be inspected of the workpiece 50, while moving the robot hand 201 gripping the workpiece 50, along a movement route including a plurality of imaging points set on the surface to be inspected so that the surface to be inspected of the workpiece 50 is entirely covered by a plurality of images imaged by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota wherein the imaging points (interpreted to be a working point) are located on the edges of the workpiece.), wherein the working point is kept at a predetermined working offset from an ancillary tool associated with an edge-following system ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”); and
cause movement of the multi-axis robot to manipulate the object via the handling tool while maintaining the predetermined working offset continuously between the ancillary tool and the working point ([0038] via “In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … Thus, the accuracy of the flaw inspection can be improved by, in addition to the imaging in which the imaging region including the imaging point is perpendicular to the optical axis of the camera 210 (and the illumination light of the illumination 220), for example, as shown in FIG. 4C, adjusting the orientation of the workpiece 50 gripped by the robot hand 201, by the operation of the robot hand 201, for example, as shown in FIG. 4D or FIG. 4E, so that, in the same imaging point, the imaging region including the imaging point has an angle that is not perpendicular to the optical axis of the camera 210 and the illumination light of the illumination 220. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50
gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”), ([0041] via “The movement operation control unit 330 moves the robot hand 201 on the basis of the movement route of the robot hand 201 calculated by the imaging position information setting unit 310. Thereby, the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is controlled so that all imaging points included in the imaging position information set by the imaging position information setting unit 310 are covered by the imaging points where imaging is performed by the camera 210.”).
Oota is silent on to determine one or more dimensional attributes associated with the object; cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side.
However, Diankov teaches to determine one or more dimensional attributes associated with the object ([0041] via “In some embodiments, the destination crossing sensor 316 can be used to measure a height of the target object 112 during transfer. … The robotic system 100
can compare the gripper height 322 to a crossing reference height 324 (e.g., a known vertical position of the destination crossing sensor 316 and/or a reference line/plane thereof) to calculate an object height 320 of the target object 112 that is being transferred.”);
cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330. Also, the robotic system 100 can derive the scanning maneuver 404 for rotating the target object 112 to present multiple surfaces thereof to the scanning sensor 330.”), ([0056] via “Also, as an illustrative example, the robotic system 100 can derive scanning maneuver 414 for moving the recognized object along the x-axis and/or the y-axis and/or for rotating the object about the z-axis.”), (Note: See Figures 4A-4D of Diankov as well.);
wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side ([0024] via “In some embodiments, the task can include manipulation (e.g., moving and/or reorienting) of a target object 112 (e.g., one of the packages, boxes, cases, cages, pallets, etc. corresponding to the executing task) from a start location 114 to a task location 116.”), ([0053] via “The robotic system 100 can further derive the scanning maneuver 414 based on the estimates of the corner and/or the end portion locations. For example, the robotic system 100 can derive the scanning maneuver 414 for horizontally/vertically displacing the target object 112 to present multiple corners and/or end portions thereof to the scanning sensor 330.”), (Note: See Figures 4A-4D and 6A-6F of Diankov as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diankov wherein the at least one processor is configured to: determine one or more dimensional attributes associated with the object; cause movement of the multi-axis robot to manipulate the object via the handling tool such that the working point is configured to move along the edge of the object from a first location on a first side of the edge of the object to a second location along a second side of the edge of the object; wherein the edge of the object comprises surfaces along a plurality of sides connected with corners, and wherein the plurality of sides include the first side and the second side. Doing so allows the system to accurately recognize all dimensions and surfaces of the object for further object processing, as stated by Diankov ([0054] via “The robotic system 100
can operate the scanning sensor 330 based on placing the end effector 304 at the scanning position 412 and/or based on implementing the scanning maneuver 414. … Thus, based on the scanning position 412 and/or the scanning maneuver 414, the robotic system 100 can present multiple surfaces/portions of the unrecognized objects and increase the likelihood of accurately locating and scanning identifiers on the unrecognized objects.”).
9. Claim(s) 2, 4, 5, 12, 14, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov), and further in view of Shivaram et al. (US 20170024613 A1 hereinafter Shivaram).
Regarding Claim 2, modified reference Oota teaches the computer-implemented method of claim 1, wherein the ancillary tool is an image capturing device ([0036] via “The camera 210 is an imaging means of imaging the surface to be inspected of the workpiece 50, and, for example, is composed of an imaging element such as a CCD image sensor and a CMOS image sensor. The camera 220 is supported in a predetermined posture so that the surface to be inspected of the workpiece 50 gripped by the robot hand 201 can be imaged, by a support body 213.”), the computer-implemented method further comprising:
capturing, via the image capturing device, image data associated with the edge of the object during execution of the edge-following operation ([0037] via “The control device 300
causes the camera 210 to image in each imaging point set on the surface to be inspected of the workpiece 50, while moving the robot hand 201 gripping the workpiece 50, along a movement route including a plurality of imaging points set on the surface to be inspected so that the surface to be inspected of the workpiece 50 is entirely covered by a plurality of images imaged by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota wherein the image data comprises the edges of the workpiece.).
Oota is silent on generating, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side.
However, Shivaram teaches generating, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side ([0053] via “Reference is now made to
FIG. 7, which shows a procedure 700 for generating stitched and composite images of workpieces in each station. In step 710, the procedure 700 computes a region imaged by each camera from each station in the common coordinate system that was determined above. In step 720, a bounding box is computed that contains all the regions. In step 730, the procedure 700 creates two stitched images, one for each station.”), ([0058] via “A graphical user interface (GUI) display with similar depictions is shown in FIGS. 12-15. In FIG. 12, a four-camera arrangement generates four respective images 1210, 1220, 1230 and 1240 of the first workpiece placed on (e.g.) a pick platform. Note that the respective edge 1212, 1222, 1232 and 1242 of the workpiece is scaled differently in each image due to the physical placement of the respective camera relative to the platform and workpiece.”), (Note: See Figures 7 and 12-15 of Shivaram as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Shivaram wherein the computer-implemented method further comprises: generating, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side. When the field of view of the camera is too small to fully capture the object, creating a continuous image allows a larger desired field of view of the object to be captured, as stated by Shivaram ([0054] via “As used herein, the term “stitched image” or “stitching” relates to a process that combines two or more source images into one composite result image. The process is useful when a camera field of view is too small to capture the entire desired scene and multiple images are required.”).
Regarding Claim 4, modified reference Oota teaches the computer-implemented method of claim 2, wherein the predetermined working offset is defined at least in part by a predetermined focal length of a lens of the image capturing device ([0038] via “As shown in
FIG. 4A, the imaging point means a point located on the optical axis of when the imaging is performed by the camera 210, and the imaging region means an imaging range imaged by the camera 210. When a distance from the camera 210 to the imaging point is snort, the imaging region is small (the field of view is small) as in an imaging region 1 shown in FIG. 4B. When the distance from the camera 210 to the imaging point is long, the imaging region is large (the field of view is large) as in an imaging region 2. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand
201, and the optical axis of the camera 210 and the illumination light of the illumination 220
is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”).
Regarding Claim 5, modified reference Oota teaches the computer-implemented method of claim 2, the edge-following operation further comprising: determining a working orientation of the working point of the object relative to the image capturing device after causing the handling tool to engage the object, wherein the working orientation and working offset are configured to be controlled by the handling tool of the multi-axis robot such that the working point on the edge of the object remains in focus of the image capturing device during the edge-following operation ([0038] via “As shown in FIG. 4A, the imaging point means a point located on the optical axis of when the imaging is performed by the camera 210, and the imaging region means an imaging range imaged by the camera 210. When a distance from the camera 210 to the imaging point is snort, the imaging region is small (the field of view is small) as in an imaging region 1 shown in FIG. 4B. When the distance from the camera 210 to the imaging point is long, the imaging region is large (the field of view is large) as in an imaging region 2. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. When the surface to be inspected of the workpiece 50 is imaged by the camera 210, depending on a shape of a flaw formed on the surface to be inspected of the workpiece 50, a plurality of positional relationships of the camera 210 and the illumination 220, and the imaging point of the workpiece 50 need to be set. Thus, the accuracy of the flaw inspection can be improved by, in addition to the imaging in which the imaging region including the imaging point is perpendicular to the optical axis of the camera 210 (and the illumination light of the illumination 220), for example, as shown in FIG. 4C, adjusting the orientation of the workpiece 50 gripped by the robot hand 201, by the operation of the robot hand 201, for example, as shown in FIG. 4D or FIG. 4E, so that, in the same imaging point, the imaging region including the imaging point has an angle that is not perpendicular to the optical axis of the camera 210 and the illumination light of the illumination 220. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50
gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”).
Regarding Claim 12, modified reference Oota teaches the edge-following system of claim 11, wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: capture, via an image capturing device, image data associated with the edge of the object ([0037] via “The control device 300 causes the camera 210 to image in each imaging point set on the surface to be inspected of the workpiece 50, while moving the robot hand 201 gripping the workpiece 50, along a movement route including a plurality of imaging points set on the surface to be inspected so that the surface to be inspected of the workpiece 50 is entirely covered by a plurality of images imaged by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota wherein the image data comprises the edges of the workpiece.).
Oota is silent on to generate, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side.
However, Shivaram teaches to generate, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side ([0053] via “Reference is now made to
FIG. 7, which shows a procedure 700 for generating stitched and composite images of workpieces in each station. In step 710, the procedure 700 computes a region imaged by each camera from each station in the common coordinate system that was determined above. In step 720, a bounding box is computed that contains all the regions. In step 730, the procedure 700 creates two stitched images, one for each station.”), ([0058] via “A graphical user interface (GUI) display with similar depictions is shown in FIGS. 12-15. In FIG. 12, a four-camera arrangement generates four respective images 1210, 1220, 1230 and 1240 of the first workpiece placed on (e.g.) a pick platform. Note that the respective edge 1212, 1222, 1232 and 1242 of the workpiece is scaled differently in each image due to the physical placement of the respective camera relative to the platform and workpiece.”), (Note: See Figures 7 and 12-15 of Shivaram as well.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Shivaram wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: generate, based on the image data, a continuous image of the edge of the object from the first location to the second location, including a first corner disposed between the first side and the second side. When the field of view of the camera is too small to fully capture the object, creating a continuous image allows a larger desired field of view of the object to be captured, as stated by Shivaram ([0054] via “As used herein, the term “stitched image” or “stitching” relates to a process that combines two or more source images into one composite result image. The process is useful when a camera field of view is too small to capture the entire desired scene and multiple images are required.”).
Regarding Claim 14, modified reference Oota teaches the edge-following system of claim 12, wherein the predetermined working offset is defined at least in part by a predetermined focal length of a lens of the image capturing device ([0038] via “As shown in
FIG. 4A, the imaging point means a point located on the optical axis of when the imaging is performed by the camera 210, and the imaging region means an imaging range imaged by the camera 210. When a distance from the camera 210 to the imaging point is snort, the imaging region is small (the field of view is small) as in an imaging region 1 shown in FIG. 4B. When the distance from the camera 210 to the imaging point is long, the imaging region is large (the field of view is large) as in an imaging region 2. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. … In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand
201, and the optical axis of the camera 210 and the illumination light of the illumination 220
is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210 is uniquely determined.”).
Regarding Claim 15, modified reference Oota teaches The edge-following system of claim 12, wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: determine a working orientation of the working point of the object relative to the image capturing device after causing the handling tool to engage the object, wherein the working orientation and working offset are configured to be controlled by the handling tool of the multi-axis robot such that the working point on the edge of the object remains in focus of the image capturing device ([0038] via “As shown in FIG. 4A, the imaging point means a point located on the optical axis of when the imaging is performed by the camera 210, and the imaging region means an imaging range imaged by the camera 210. When a distance from the camera 210 to the imaging point is snort, the imaging region is small (the field of view is small) as in an imaging region 1 shown in FIG. 4B. When the distance from the camera 210 to the imaging point is long, the imaging region is large (the field of view is large) as in an imaging region 2. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, the number of imaging can be increased or decreased by adjusting the distance from the camera 210 to the imaging point, within a range of focus in the imaging point. When the surface to be inspected of the workpiece 50 is imaged by the camera 210, depending on a shape of a flaw formed on the surface to be inspected of the workpiece 50, a plurality of positional relationships of the camera 210 and the illumination 220, and the imaging point of the workpiece 50 need to be set. Thus, the accuracy of the flaw inspection can be improved by, in addition to the imaging in which the imaging region including the imaging point is perpendicular to the optical axis of the camera
210 (and the illumination light of the illumination 220), for example, as shown in FIG. 4C, adjusting the orientation of the workpiece 50 gripped by the robot hand 201, by the operation of the robot hand 201, for example, as shown in FIG. 4D or FIG. 4E, so that, in the same imaging point, the imaging region including the imaging point has an angle that is not perpendicular to the optical axis of the camera 210 and the illumination light of the illumination 220. In this way, when the surface to be inspected of the workpiece 50 is imaged by the camera 210, by specifying the imaging point, the distance from the camera 210 to the imaging point, and the orientation of the workpiece 50 in the imaging point (hereinafter, these are referred to as “imaging position information”), the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is uniquely determined, and the imaging region of the surface to be inspected imaged by the camera 210
is uniquely determined.”).
10. Claim(s) 3 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov), further in view of Shivaram et al. (US 20170024613 A1 hereinafter Shivaram), and further in view of Bufi et al. (US 20220366558 A1 hereinafter Bufi).
Regarding Claim 3, modified reference Oota teaches the computer-implemented method of claim 2, but is silent on the computer-implemented method further comprising: inputting the continuous image into an anomaly detection model; and detecting, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object.
However, Bufi teaches inputting the continuous image into an anomaly detection model; and detecting, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object ([0012] via “Provided is a system for visual inspection of an article. The system includes a camera for acquiring image data of an article under inspection, a node computing device for receiving the image data from the camera and analyzing the image data using a defect detection model trained to detect at least one defect type, the defect detection model comprising a machine-learning based object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output, and a programmable logic controller (“PLC”) device for receiving the defect data from the node computing device and determining whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Bufi wherein the computer-implemented method further comprises: inputting the continuous image into an anomaly detection model; and detecting, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object. Doing so incorporates an algorithm that is able to autonomously detect a wide range of defects of imaged articles, as stated by Bufi ([0208] via “The defect detection model 154 is a machine learning-based model configured to analyze image data of the article 110 and identify defects therein. … The defect detection model 154 may be configured to perform multiclass classification for classifying instances into one of three or more classes. Classes may correspond to different defect types that the defect detection model 154 has been trained to detect.”).
Regarding Claim 13, modified reference Oota teaches the edge-following system of claim 12, but is silent on wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: input the continuous image into an anomaly detection model; and detect, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object.
However, Bufi teaches to input the continuous image into an anomaly detection model; and detect, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object ([0012] via “Provided is a system for visual inspection of an article. The system includes a camera for acquiring image data of an article under inspection, a node computing device for receiving the image data from the camera and analyzing the image data using a defect detection model trained to detect at least one defect type, the defect detection model comprising a machine-learning based object detection model configured to receive the image data as an input and generate defect data describing a detected defect as an output, and a programmable logic controller (“PLC”) device for receiving the defect data from the node computing device and determining whether the defect data is acceptable or unacceptable by comparing the defect data to tolerance data.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Bufi wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: input the continuous image into an anomaly detection model; and detect, based on an output generated by the anomaly detection model, one or more physical defects associated with the edge of the object. Doing so incorporates an algorithm that is able to autonomously detect a wide range of defects of imaged articles, as stated by Bufi ([0208] via “The defect detection model 154 is a machine learning-based model configured to analyze image data of the article 110 and identify defects therein. … The defect detection model 154 may be configured to perform multiclass classification for classifying instances into one of three or more classes. Classes may correspond to different defect types that the defect detection model 154 has been trained to detect.”).
11. Claim(s) 6, 10, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov), and further in view of DeFant et al. (US 10821611 B1 hereinafter DeFant).
Regarding Claim 6, modified reference Oota teaches the computer-implemented method of claim 1, but is silent on wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object.
However, DeFant teaches wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object (Col. 7 lines 26-35, where “Before the multi-zone end effector assembly 103 engages with the package 106, the end effector application 132 (FIG. 1B) can use the image sensor 128 (FIG. 1B) to identify that the package
106 has a label 109 and its location on the package 106 in some cases. In some non-limiting examples, the end effector application 132 can determine the dimensions of the package
106 using various computer vision algorithms. The determined dimensions may include a height, width, length, and/or other suitable package dimensions.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of DeFant wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object. Knowing the dimensions of the object allows the robot to optimize engagement of the object, as stated by DeFant (Col. 6 lines 9-13, where “The number of zones 235 activated may be based on the dimensions of the package 106. For example, a package 106 with dimensions that are less than a threshold may be attached with one or two zones 235 of compression cups 212.”).
Regarding Claim 10, modified reference Oota teaches the computer-implemented method of claim 1, wherein the edge-following system is configured to perform the edge-following operation based solely on the exterior dimensions ([0041] via “The movement operation control unit 330 moves the robot hand 201 on the basis of the movement route of the robot hand 201 calculated by the imaging position information setting unit 310. Thereby, the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is controlled so that all imaging points included in the imaging position information set by the imaging position information setting unit 310 are covered by the imaging points where imaging is performed by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota, wherein all of the imaging points are along the exterior dimensions of the workpiece. Additionally, [0053] and [0056] and Figures 4A-4D and 6A-6F of Diankov also inspect the package along its exterior.).
Oota is silent on wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model.
However, DeFant teaches wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model (Col. 7 lines 26-35, where “Before the multi-zone end effector assembly 103 engages with the package 106, the end effector application 132 (FIG. 1B) can use the image sensor 128 (FIG. 1B) to identify that the package 106 has a label 109 and its location on the package 106 in some cases. In some non-limiting examples, the end effector application 132 can determine the dimensions of the package 106 using various computer vision algorithms. The determined dimensions may include a height, width, length, and/or other suitable package dimensions.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of DeFant wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model. Knowing the dimensions of the object allows the robot to optimize engagement of the object, as stated by DeFant (Col. 6 lines 9-13, where “The number of zones 235 activated may be based on the dimensions of the package 106. For example, a package 106 with dimensions that are less than a threshold may be attached with one or two zones 235 of compression cups 212.”).
Regarding Claim 16, modified reference Oota teaches the edge-following system of claim 11, but is silent on wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object.
However, DeFant teaches wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object (Col. 7 lines 26-35, where “Before the multi-zone end effector assembly 103 engages with the package 106, the end effector application 132 (FIG. 1B) can use the image sensor 128 (FIG. 1B) to identify that the package
106 has a label 109 and its location on the package 106 in some cases. In some non-limiting examples, the end effector application 132 can determine the dimensions of the package
106 using various computer vision algorithms. The determined dimensions may include a height, width, length, and/or other suitable package dimensions.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of DeFant wherein the one or more dimensional attributes associated with the object are determined via an object localization vision model, and wherein the one or more dimensional attributes comprise at least one of a length, a width, a depth, a corner location, a corner radius (e.g., radius of curvature), a center, an area, a position, an orientation, or any other external physical characteristics of the object. Knowing the dimensions of the object allows the robot to optimize engagement of the object, as stated by DeFant (Col. 6 lines 9-13, where “The number of zones 235 activated may be based on the dimensions of the package 106. For example, a package 106 with dimensions that are less than a threshold may be attached with one or two zones 235 of compression cups 212.”).
Regarding Claim 20, modified reference Oota teaches the edge-following system of claim 11, wherein the edge-following system is configured to use solely the exterior dimensions to define the working point ([0041] via “The movement operation control unit 330 moves the robot hand 201 on the basis of the movement route of the robot hand 201 calculated by the imaging position information setting unit 310. Thereby, the positional relationship of the surface to be inspected of the workpiece 50 gripped by the robot hand 201, and the optical axis of the camera 210 and the illumination light of the illumination 220 is controlled so that all imaging points included in the imaging position information set by the imaging position information setting unit 310 are covered by the imaging points where imaging is performed by the camera 210.”), (Note: See Figures 4A-4E and 7 of Oota, wherein all of the imaging points (i.e., working points) are along the exterior dimensions of the workpiece.).
Oota is silent on wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model.
However, DeFant teaches wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model (Col. 7 lines 26-35, where “Before the multi-zone end effector assembly 103 engages with the package 106, the end effector application 132 (FIG. 1B) can use the image sensor 128 (FIG. 1B) to identify that the package 106 has a label 109 and its location on the package 106 in some cases. In some non-limiting examples, the end effector application 132 can determine the dimensions of the package 106 using various computer vision algorithms. The determined dimensions may include a height, width, length, and/or other suitable package dimensions.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of DeFant wherein the one or more dimensional attributes comprise exterior dimensions of the object as measured by an object localization vision model. Knowing the dimensions of the object allows the robot to optimize engagement of the object, as stated by DeFant (Col. 6 lines 9-13, where “The number of zones 235 activated may be based on the dimensions of the package 106. For example, a package 106 with dimensions that are less than a threshold may be attached with one or two zones 235 of compression cups 212.”).
12. Claim(s) 7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov), and further in view of Abe et al. (US 20250086341 A1 hereinafter Abe).
Regarding Claim 7, modified reference Oota teaches the computer-implemented method of claim 1, but is silent on wherein executing the edge-following operation further comprises: determining, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object.
However, Abe teaches determining, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object ([0065] via “As illustrated in FIG. 8, maintain displacements include cases in which the main target object 90A is translated while face contact is maintained, and include cases in which the main target object 90A is rotated while one-edge contact or two-edge contact is maintained. Separation displacements are displacements in which the main target object 90A is displaced in a direction of increased degrees of freedom of contact state. As illustrated in FIG. 8, separation displacements include translating the main target object 90A in a direction that intersects with a contact face of face contact with the auxiliary target object 90B that is a direction away from the auxiliary target object 90B. Moreover, separation displacements include rotating the main target object 90A to give one-edge contact from face contact or from two-edge contact.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Abe wherein executing the edge-following operation further comprises: determining, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object. Doing so maintains the position of the object relative to its environment while rotating the object, as stated above by Abe.
Regarding Claim 17, modified reference Oota teaches the edge-following system of claim 11, but is silent on wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: determine, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object.
However, Abe teaches to determine, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object ([0065] via “As illustrated in FIG. 8, maintain displacements include cases in which the main target object
90A is translated while face contact is maintained, and include cases in which the main target object 90A is rotated while one-edge contact or two-edge contact is maintained. Separation displacements are displacements in which the main target object 90A is displaced in a direction of increased degrees of freedom of contact state. As illustrated in FIG. 8, separation displacements include translating the main target object 90A in a direction that intersects with a contact face of face contact with the auxiliary target object 90B that is a direction away from the auxiliary target object 90B. Moreover, separation displacements include rotating the main target object 90A to give one-edge contact from face contact or from two-edge contact.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Abe wherein the computer-coded instructions, when executed by the at least one processor, further cause the edge-following system to: determine, based at least in part on the one or more dimensional attributes, one or more points of rotation associated with the object, wherein the one or more points of rotation are associated with a corner radius of the object. Doing so maintains the position of the object relative to its environment while rotating the object, as stated above by Abe.
13. Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Oota et al. (US 20180370027 A1 hereinafter Oota) in view of Diankov et al. (US 20200130963 A1 hereinafter Diankov), and further in view of Namiki (US 20190184582 A1 hereinafter Namiki).
Regarding Claim 9, modified reference Oota teaches the computer-implemented method of claim 1, but is silent on wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object.
However, Namiki teaches wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object ([0074] via “The image control unit
51 sends the detected position of the workpiece 38 to the operation control unit 43. On the basis of the obtained position of the workpiece 38, the operation control unit 43 corrects the position and the orientation of the robot 1 set in the operation program 41. In other word, the position and the orientation of the hand 5 when the workpiece 38 is gripped is corrected. Thus, the operation control unit 43 can drive the robot 1 and the hand 5 so as to grip the workpiece 38.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Namiki wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object. Doing so enables the robot to locate and properly engage the object, as stated above by Namiki.
Regarding Claim 19, modified reference Oota teaches the edge-following system of claim 11, but is silent on wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object.
However, Namiki teaches wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object ([0074] via “The image control unit
51 sends the detected position of the workpiece 38 to the operation control unit 43. On the basis of the obtained position of the workpiece 38, the operation control unit 43 corrects the position and the orientation of the robot 1 set in the operation program 41. In other word, the position and the orientation of the hand 5 when the workpiece 38 is gripped is corrected. Thus, the operation control unit 43 can drive the robot 1 and the hand 5 so as to grip the workpiece 38.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Namiki wherein causing the handling tool associated with the multi-axis robot to engage the object further comprises engaging the object based in part on the one or more dimensional attributes associated with the object. Doing so enables the robot to locate and properly engage the object, as stated above by Namiki.
Examiner’s Note
14. The Examiner has cited particular paragraphs or columns and line numbers in the
references applied to the claims above for the convenience of the Applicant. Although the
specified citations are representative of the teachings of the art and are applied to specific
limitations within the individual claim, other passages and figures may apply as well. It is
respectfully requested of the Applicant in preparing responses, to fully consider the references
in their entirety as potentially teaching all or part of the claimed invention, as well as the
context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP
2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole,
including portions that would lead away from the claimed Invention. W.L. Gore & Associates,
Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851
(1984). See also MPEP §2123.
Conclusion
15. Any inquiry concerning this communication or earlier communications from the
examiner should be directed to BYRON X KASPER whose telephone number is (571)272-3895.
The examiner can normally be reached Monday - Friday 8 am - 5 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Adam Mott can be reached on (571) 270-5376. The fax phone number for the
organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be
obtained from Patent Center. Unpublished application information in Patent Center is available
to registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for
more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BYRON XAVIER KASPER/Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights