DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Joint Inventors
This application currently names joint inventors. In considering patentability of the claims, the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the Examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/21/2026 has been entered.
Status of Claims
This action is in response to Applicant’s Request for Continued Examination filed on 07/21/2026. Claims 1-19 are pending and examined below.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 4, 7, 11, 14, 16, and 19 are rejected under 35 U.S.C. 103 as being obvious over Higo, US 20190308325 A1, herein referred to as Higo, and in view of Sato et al, US 20120253512 A1, herein referred to as Sato.
Regarding claim 1, Higo discloses determining a current position and an attitude of a visual sensor of the robot relative to the 3D object (Paragraph 0029, 0048; imaging capturing unit is moved to a given location and has a specific orientation to capture the 3D objects, position and orientation of the 3D object is determined relative to the image capturing unit), acquiring a grabbing template of the 3D object, the grabbing template comprising a specified grabbing position and attitude of the visual sensor relative to the 3D object (Paragraphs 0029-0030, 0045, 0070; a suction target map is generated to determine specific suction zones on each object which correspond to specific positioning and orientation of the objects relative to the imaging unit, *examiner notes that under BRI, grabbing of an object can include vacuum suction of an object), judging whether the grabbing template further comprises at least one reference grabbing position and attitude of the visual sensor relative to the 3D object, wherein the reference grabbing position and attitude is generated on the basis of the specified grabbing position and attitude (Paragraphs 0029-0030, 0045; suction target map can contain areas of the object that can be suctioned, these areas that can be suctioned have their position and orientation known relative to the camera), based on a judgment result, using the grabbing template and the current position and attitude to generate a grabbing position and attitude of the robot (Paragraph 0058; based on the suction target map, the robot actuates to the appropriate suction position), and actuating the robot to move to the generated grabbing position and attitude and grab the object (Paragraph 0058; based on the suction target map, the robot actuates to the appropriate suction position to grasp the object), but fails to disclose a fingered robot, wherein the fingered robot includes at least two fingers, the grabbing position includes two positions located on edges of the object, and each of the at least two fingers of the robot contacts a respective one of the two positions.
However, Sato, in an analogous field of endeavor, teaches a fingered robot, wherein the fingered robot includes at least two fingers (at least Figs. 5, 7; the robot may have multiple fingers for grasping an object), the grabbing position includes two positions located on edges of the object (at least Figs. 6, 8, Paragraphs 0074, 0087-0089; the robot may contact multiple gabbing surfaces of an object which can be considered as edges; these edges have given positions on the object), and each of the at least two fingers of the robot contacts a respective one of the two positions (at least Figs. 6, 8, Paragraphs 0074, 0087-0089; the fingers of the robot may contact two separate edges for manipulation of the object). Therefore, from the teaching of Sato, it would have been obvious to one of ordinary skill in the art before the effective filing date to have modified, with a reasonable expectation for success, the robotic system of Higo to include a fingered robot, wherein the fingered robot includes at least two fingers, the grabbing position includes two positions located on edges of the object, and each of the at least two fingers of the robot contacts a respective one of the two positions, as taught/suggested by Sato. The motivation to do so would be to utilize a well-known robotic manipulator variant to grasp objects on two edges of an object. Additionally, ensuring that the fingers contact edge positions of the object can allow for more secure gripping.
Regarding claim 4, Higo in view of Sato renders obvious all the limitations of claim 1. Higo further discloses the generating the grabbing template (Paragraph 0030; the suction target map is generated).
Regarding claim 7, Higo in view of Sato renders obvious all the limitations of claim 1. Higo further discloses acquiring a real image of the 3D object at a visual angle of a current position and attitude (Paragraph 0054; imaging unit may obtain an image of the object at a given position and orientation), acquiring an image template of the 3D object, the image template representing multiple virtual images of a virtual model of the 3D object at visual angles of multiple different positions and attitudes (Paragraph 0054; an image is compared to a template to determine similarities; the template is obtained by imaging an object in a multitude of positions and orientations, this can be considered a virtual model comprising virtual images as the template is stored), determining a degree of similarity between the real image and each of the multiple virtual images respectively (Paragraph 0054; similarities between the image template and the actual image may be determined), and generating the current position and attitude based on a corresponding position and attitude of a virtual image with the highest degree of similarity (Paragraph 0054; positioning and orientation may be determined based on image features matching the template, matching between two data points indicates the highest degree of similarity).
Regarding claims 11 and 14, the claim limitations are similar to those in claims 1 and 4 and are rejected using the same rationale as seen above in claims 1 and 4.
Regarding claim 16, the claim limitations are similar to those in claim 7 and are rejected using the same rationale as seen above in claim 7.
Regarding claim 19, a portion of the claim limitations are similar to those in claim 1 and are rejected using the same rationale as seen above in claim 1. Additionally, Higo discloses a processor (Fig. 3 item H11; system includes a CPU), and a memory (Fig. 3 item H14; system includes memory storage).
Claims 2-3 and 12-13 are rejected under 35 U.S.C. 103 as being obvious over Higo, in view of Sato, and further in view of Zhang et al., CN 110076662 A, herein referred to as Zhang.
Regarding claim 2, Higo in view of Sato renders obvious all the limitations of claim 1. Higo further discloses based on the judgment result, using the grabbing template and the current position and attitude to determine a target grabbing position and attitude of the visual sensor relative to the 3D object (Paragraph 0058; based on the suction target map, the robot actuates to the appropriate suction position), and converting the target grabbing position and attitude to the grabbing position and attitude of the robot by calibration (Paragraph 0085; calibration may be performed in advance to determine positional and orientational relationships between the imaging unit and the robot), but fails to disclose converting the target grabbing position and attitude to the grabbing position and attitude of the robot by hand-eye calibration. However, Zhang, in an analogous field of endeavor, teaches converting the target grabbing position and attitude to the grabbing position and attitude of the robot by hand-eye calibration (Paragraph 0118; a hand-eye Jacobian matrix may be used to establish a relationship between the camera and the robot). Therefore, from the teaching of Zhang, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the robotic system of Higo and Sato to include converting the target grabbing position and attitude to the grabbing position and attitude of the robot by hand-eye calibration, as taught/suggested by Zhang. The motivation to do so would be to use a well-known method for calibration of the robot to determine spatial relationships between the imaging unit and the robot.
Regarding claim 3, Higo, in view of Sato, and further in view of Zhang renders obvious all the limitations of claim 2. Higo further discloses when the grabbing template further comprises at least one said reference grabbing position and attitude, determining a grabbing position and attitude with the shortest movement distance from the current position and attitude in the grabbing template, to serve as the target grabbing position and attitude (Paragraphs 0030 and 0055; the object with the smallest depth value obtained from the imaging unit is selected for suctioning), and when the grabbing template does not comprise the reference grabbing position and attitude, taking the specified grabbing position and attitude to be the target grabbing position and attitude (Paragraphs 0030 and 0055; the suction target map may be used to determine if an object has an area suitable for suctioning; if an object does not have a suitable area, the closest object which does have a suitable suction area is selected, such as those at the side of a pile which might not interfere with other objects).
Regarding claims 12-13, the claim limitations are similar to those in claims 2-3 and are rejected using the same rationale as seen above in claims 2-3.
Claims 5-6, 10, 15, and 18 are rejected under 35 U.S.C. 103 as being obvious over Higo, in view of Sato, and further in view of Tateno et al., US 20130245828 A1, herein referred to as Tateno.
Regarding claim 5, Higo in view of Sato renders obvious all the limitations of claim 4. Higo further discloses generating the grabbing template (Paragraph 0030; a suction target map is generated), but fails to disclose acquiring a specified virtual image of a virtual model of the 3D object at a visual angle of the specified grabbing position and attitude, simulating multiple different positions and attitudes of the visual sensor relative to the 3D object, and obtaining multiple virtual images of the virtual model of the 3D object at visual angles of the multiple different positions and attitudes, determining a degree of similarity between each of the multiple virtual images and the specified virtual image respectively, and taking a corresponding position and attitude of a virtual image with a degree of similarity higher than a preset threshold to be the reference grabbing position and attitude.
However, Tateno, in an analogous field of endeavor, teaches acquiring a specified virtual image of a virtual model of the 3D object at a visual angle of the specified grabbing position and attitude (Paragraph 0065; a virtual image may be obtained of a virtual model of an object), simulating multiple different positions and attitudes of the visual sensor relative to the 3D object, and obtaining multiple virtual images of the virtual model of the 3D object at visual angles of the multiple different positions and attitudes (Fig. 4, Paragraph 0065; multiple virtual images from different viewing positions and orientations may be obtained by the virtual imaging sensor), determining a degree of similarity between each of the multiple virtual images and the specified virtual image respectively (Paragraphs 0111-0112; viewpoints most similar to a determined virtual imaging viewpoint may be selected based on differences between the positioning and orientation), and taking a corresponding position and attitude of a virtual image with a degree of similarity higher than a preset threshold to be the reference grabbing position and attitude (Paragraphs 0111-0112; position and orientation similarities may be ranked based on the differences themselves; those with the lowest rank sum are selected; the ranks themselves may be considered pre-set thresholds as they are determined before selection).
Therefore, from the teaching of Tateno, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the robotic system of Higo and Sato to include acquiring a specified virtual image of a virtual model of the 3D object at a visual angle of the specified grabbing position and attitude, simulating multiple different positions and attitudes of the visual sensor relative to the 3D object, and obtaining multiple virtual images of the virtual model of the 3D object at visual angles of the multiple different positions and attitudes, determining a degree of similarity between each of the multiple virtual images and the specified virtual image respectively, and taking a corresponding position and attitude of a virtual image with a degree of similarity higher than a preset threshold to be the reference grabbing position and attitude, as taught/suggested by Tateno. The motivation to do so would be to increase the accuracy of the system by evaluating different viewpoints of objects to be grabbed. This can allow the system to determine which objects have a higher likelihood of 1) being the correct object to be grasped and 2) correctly being grasped.
Regarding claim 6, Higo, in view of Sato, and further in view of Tateno renders obvious all the limitations of claim 5. Higo further discloses generating the grabbing template (Paragraph 0030; a suction target map is generated), but fails to disclose the degree of similarity comprises a degree of similarity between a characteristic of the virtual image and a characteristic of the specified virtual image. However, Tateno further teaches the degree of similarity comprises a degree of similarity between a characteristic of the virtual image and a characteristic of the specified virtual image (Paragraphs 0111-0112; viewpoints most similar to a determined virtual imaging viewpoint may be selected based on differences between the positioning and orientation; positioning and orientation of the viewpoints may be considered characteristics relative to an image taken at those viewpoints). Therefore, from the teaching of Tateno, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the robotic system of Higo, Sato, and Tateno to include the degree of similarity comprises a degree of similarity between a characteristic of the virtual image and a characteristic of the specified virtual image, as taught/suggested by Tateno. The motivation to do so would be to allow the system to determine the best orientation of the robot for grabbing the object.
Regarding claim 10, Higo in view of Sato renders obvious all the limitations of claim 1. Higo further discloses establishing an Earth model with a specified grabbing point on a virtual model of the 3D object, wherein the current position and attitude, the specified grabbing position and attitude and the reference grabbing position and attitude are represented by position and attitude parameters in the Earth model (Paragraphs 0045, 0056; 6 DoF coordinate system may be established for the robot and imaging unit; objects may have an associated target suction map which indicates positions that the robot can suction), but fails to disclose establishing an Earth model with a specified grabbing point on a virtual model of the 3D object as a sphere center. However, Tateno teaches establishing an Earth model with a specified grabbing point on a virtual model of the 3D object as a sphere center (Fig. 4, Paragraph 0065; an object may be at the center of a virtual sphere for establishing viewpoints of the imaging unit; viewpoints have given positioning and orientation relative to the object). Therefore, from the teaching of Tateno, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the robotic system of Higo and Sato to include establishing an Earth model with a specified grabbing point on a virtual model of the 3D object as a sphere center, as taught/suggested by Tateno. The motivation to do so would be to establish positioning and orientation around a given point. This can allow for easier representations in a given frame of reference.
Regarding claim 15, the claim limitations are similar to those in claim 5 and are rejected using the same rationale as seen above in claim 5.
Regarding claim 18, the claim limitations are similar to those in claim 10 and are rejected using the same rationale as seen above in claim 10.
Claims 8-9 and 17 are rejected under 35 U.S.C. 103 as being obvious over Higo, in view of Sato, and further in view of Lee et al., US 20210101286 A1, herein referred to as Lee.
Regarding claim 8, Higo in view of Sato renders obvious all the limitations of claim 7. Higo further discloses using a masking model to generate a mask of the 3D object based on the real image of the 3D object (Paragraph 0056; a mask may be applied to an image of the 3D object), and obtaining a characteristic of the real image of the 3D object based on the mask of the 3D object (Paragraph 0056; the mask of the 3D object results in areas that are suitable for suction; these areas correspond to pixels within the image mask), but fails to disclose using a Mask-RCNN model to generate a mask of the 3D object based on the real image of the 3D object. However, Lee, in an analogous field of endeavor, teaches using a Mask-RCNN model to generate a mask of the 3D object based on the real image of the 3D object (Paragraphs 0007-0008; an RCNN model may be used to generate a mask for object detection for grasping). Therefore, from the teaching of Lee, it would have been obvious to one of ordinary skill in the art before the effective filing date to have further modified, with a reasonable expectation for success, the robotic system of Higo and Sato to include using a Mask-RCNN model to generate a mask of the 3D object based on the real image of the 3D object, as taught/suggested by Lee. The motivation to do so would be to use a well-known method for object determination using image masking.
Regarding claim 9, Higo, in view of Sato, and further in view of Lee renders obvious all the limitations of claim 8. Higo further discloses the degree of similarity comprises a degree of similarity between the characteristic of the real image and a characteristic of the virtual image (Paragraph 0054; an image is compared to a template to determine similarities; the template comprises virtual images as the template is stored; similarities may be determined based on image features).
Regarding claim 17, the claim limitations are similar to those in claim 8 and are rejected using the same rationale as seen above in claim 8.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 11, and 19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER ALLEN BUKSA whose telephone number is (571)272-5346. The examiner can normally be reached M-F 7:30 AM-4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thomas Worden can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER A BUKSA/Examiner, Art Unit 3658
Prosecution Insights