DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 13 April 2026 have been fully considered but they are not persuasive.
Applicant argues that Chu does not perform regression.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., regression) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant also argues that Chu’s orientation of the grasp rectangle in Fig. 2a-b does not correspond to a “correction amount” because the correction amount is not something that is classified but rather is output based on the first patch image (essentially a regression value). Again, regression is not claimed and the broadly recited “correction amount” does not distinguish from Chu, including Fig. 2 and Section III Problem Statement. The target range corresponds to the grasp representation in Fig. 2(a) in which the graph representation (target range) describes the location, orientation, and opening distance of a parallel plate gripper and is well within the BRI of “target range” as defined in [0104] of the instant specification as published. The “correction amount” is shown in Fig. 2(b) and is the graph rectangle which has been oriented (corrected) as further discussed in Section III. As to plural components stored in a container see Fig. 1 while noting that Chu is directed to solving the same problem of picking objects from a container whose orientations are random as further discussed in Section I Introduction.
Further in regards to the “regression arguments” Chu also applies regression in the Grasp Proposal Network and Multigrasp Detection that identifies/predicts potential grasp regions including regression over bounding box parameters from RGB-D data see sections III and IV.
In regards to amended claim 10, the arguments and amendments have been carefully considered and subjected to an update search/analysis. Based on this analysis, the features of amended claim 10 in combination with base claim 1 are also allowable over the prior art of record. The depth-weighted grayscale image component image and its use in the path image generator, in combination with the remaining limitations of claim 1 are not disclosed or fairly suggested by the prior art of record.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
an image acquirer, an image compositor, and a patch image generator of claims 17-18 which are considered nonce terms and/or otherwise fail to denote structure to one of ordinary skill in the art; moreover these elements do not recite structural elements for performing the recited functions.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 1, 8, and 11-13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chu {F. J. Chu, R. Xu, and P. A. Vela, “Real-world multiobject, multigrasp detection,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3355–3362, Oct. 2018, doi: 10.1109/LRA.2018.2852777}.
Claim 1
In regards to claim 1, Chu discloses an image processing device, comprising:
a processor {see Section V Experiments and Evaluation, Fig. 3 structural diagram of the system, and Tables III-VI which are experimental results obtained by a processor implementing the disclosed methods} configured to:
output a correction amount for correcting a position of a target range with respect to one component included in a first patch image when the first patch image is input, the target range being set for the one component, out of a plurality of components included in a stored component image representing the plurality of components stored in a container, the first patch image being cut from an image within the target range
{see Fig. 2, copied below, and Section III Problem Statement. The target range corresponds to the grasp representation in Fig. 2(a) in which the graph representation (target range) describes the location, orientation, and opening distance of a parallel plate gripper and is well within the BRI of “target range” as defined in [0104] of the instant specification as published. The “correction amount” is shown in Fig. 2(b) and is the graph rectangle which has been oriented (corrected) as further discussed in Section III. As to plural components stored in a container see Fig. 1 while noting that Chu is directed to solving the same problem of picking objects from a container whose orientations are random as further discussed in Section I Introduction};
PNG
media_image1.png
326
516
media_image1.png
Greyscale
generate a second patch image including the one component, the second patch image being an image within a range obtained by correcting the target range by the correction amount and cut from the stored component image {see the bounding box grasp configuration in Section III, equation (1) and Section IV Approach including proposal bounding box in A Grasp Proposals. See also IIIC Multigrasp Detection including refining the proposal bounding box to a non-oriented grasp bounding bow (x, y, w, h)}; and
calculate a grip success probability in the case of trying to grip the one component included in the second patch image by a robot hand located in a range where the second patch image is set {see Section IIIA in which the Grasp Proposal Network outputs probability of grasp proposal. See also VB Single-Object Multi-Grasp that calculates grip success probabilities to produce a rank ordered link of grasp candidates and Section VC. See also Table V}.
Claim 8
In regards to claim 8, Chu discloses wherein: the processor is configured to calculate the grip success probability from the second patch image using a convolutional neural network {See Section I, deep convolutional neural network, Fig. 3. Section IIIA in which the Grasp Proposal Network outputs probability of grasp proposal. See also VB Single-Object Multi-Grasp and Section VII discussing the high classification performance of CNNs for improved grasp detection outcomes of the disclosed techniques.
Claim 11
In regards to claim 11, Chu discloses a component gripping system, comprising:
the image processing device according to claim 1 {see above mapping of claim}; and
a robot hand, the image processing device causing being configured to cause the robot hand to grip the component at a position determined based on the calculated grip success probability {see VD Evaluation Metric, Section VI Results in which the highest output score of all grasp candidates is chosen as the final output to achieve high accuracy. Tables I-VI, and Fig. 5c showing the robot and robot hand used for the physical grasping test}.
Claim 12
The rejection of device claim 1 above applies mutatis mutandis to the corresponding limitations of method claim 12 while noting that the rejection above cites to both device and method disclosures.
Claim 13
In regards to claim 13, Chu discloses a component gripping method, comprising:
outputting a correction amount for correcting a position of a target range with respect to one component included in a first patch image when the first patch image is input, the target range being set for the one component, out of a plurality of components included in a stored component image representing the plurality of components stored in a container, the first patch image being cut from an image within the target range {see mapping of claim 1};
generating a second patch image including the one component, the second patch image being an image in a range obtained by correcting the target range by the correction amount and cut from the stored component image {see mapping of claim 1};
calculating a grip success probability in the case of trying to grip the one component included in the second patch image by a robot hand located in a range where the second patch image is set {see mapping of claim 1}; and
causing the robot hand to grip the component at a position determined based on the grip success probability {see mapping of claim 11}.
Allowable Subject Matter
Claims 2-7, 9, 10, and 14-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is an examiner’s statement of reasons for allowance:
Although Chu discloses the broadly worded-features of claim 1, neither Chu nor the other prior art of record discloses or fairly suggests the machine learning method of claim 2 including wherein: the processor is configured to learn a relationship of the first patch image and the correction amount, using a position difference between a position determination mask representing a proper position of the component in the target range and the component included in the first patch image as training data and in combination with the remaining elements of claim 1.
Claims 3-7 and 14-20 are potentially allowable due to their dependency upon claim 2.
Although Chu discloses the broadly worded-features of claim 1, neither Chu nor the other prior art of record discloses or fairly suggests the features of claim 9 including wherein: the processor is configured to weight a feature map output from the convolutional neural network by adding an attention mask to the feature map, and the attention mask represents to pay attention to a region extending in a gripping direction in which the robot hand grips the component and passing through a center of the second patch image and a region orthogonal to the gripping direction and passing through the center of the second patch image and in combination with the remaining elements of claim 1 and intervening claim 8.
See above Response to Arguments section for reasons why claim 10 defines over the prior art of record.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
G. C. Nandi, P. Agarwal, P. Gupta and A. Singh, "Deep Learning Based Intelligent Robot Grasping Strategy," 2018 IEEE 14th International Conference on Control and Automation (ICCA), Anchorage, AK, USA, 2018, pp. 1064-1069, doi: 10.1109/ICCA.2018.8444265 disclosing machine learning for detecting grip success and selects the best rectangle (patch image) for training the robot grasp controller. See Introduction, abstract, and sections 4.2.2, 4.5, and 4.6.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights