DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This final office action is a response to the reply received on 11/20/2025. Claims 1-14 are pending. Claims 1, and 3-14 have been amended.
Response to Arguments
Applicant’s amendments overcome the previous specification objection to Paragraph [0007]’s typographical errors.
Applicant’s amendment to Claim 6 overcomes the claim objection regarding the typographical error.
Applicants’ amendments to add the control step overcomes the previous 101 rejections for Claims 1-13. However, Claim 14 s still rejected under 101. Generally, the applicant’s remarks reference different parts of the specification and also assert that the data is gathered in real time, however this is not written in the claims. The examiner recommends amending to add the control step similarly as done in amended Claim 1 to overcome the 101 rejection.
Applicant’s amendments to the claims overcome some of the previous 112(b) rejections, but others are maintained or raised in light of the amendments (see below)
Applicant’s additional arguments with respect to the claims have been considered but are not persuasive. Applicant asserts that the prior art, Nishina, does not disclose the claim limitation “the end effector model indicating an end effector area in which the end effector can exist" on page 17 of the remarks.
However, for clarification, further see ¶0066-¶0069, ¶0080, and ¶0090-¶0098 which disclose Nishina's end effector model including hand shape data HD, finger/claw geometry, and constraints that are used to determine if the multi-fingered hand H is able to be positioned at a certain spatial configuration.
Nishina’s disclosure calculates the gripping position/postures of the fingers based on the shape data and grip information, and then determines whether a configuration is allowable/does not have interference by evaluating whether the opening width of the hand is within the maximum value and whether the hand would interfere with an object. If there is a configuration that would violate the constraints, then that configuration is rejected. The configurations that do not violate the constraints are accepted and are thus the spatial configurations that the end effector can exist without interference. Therefore, the prior art, Nishina, does teach the limitation "the end effector model indicating an end effector area in which the end effector can exist".
Information Disclosure Statement
The information disclosure statement (IDS) received on 09/10/2025 includes only the primary prior art reference relied upon in the non-final office action:
Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS dated 02/01/2024).
Thus, the IDS has been annotated and considered.
Claim Objections
Claim 3 objected to because of the following informalities: “an rear side area”. This should read “a rear side area”. Appropriate correction is required.
Claim 9 is objected to because of the following informalities: “the holders comprises at least two members…” should read “comprise at least two”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claims are generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors. It appears that the claims might include terms/phrases from a translation which makes the scope of the claims difficult to ascertain. A full review of the claims is requested, some of the examples of indefiniteness are listed below:
Amended Claims 1-14 are still rejected because it is still unclear what the scope of "holder" or “holders” (plural) is. Claim 1 appears to reference the arbitrary opening width of the holder which suggests that the opening width is the width between two claws or fingers. However, Claim 5 then references the “holder” or “each of the holders” of the end effector, which introduces ambiguity. It is not clear what the scope of “holder” or “each of the holders” entails. It seems like the holder could be one singular claw/finger (single gripping member), multiple claws/fingers/members, or could be the entire gripping tool. Furthermore, Claim 9 states that the end effector comprises holders (plural) and that the holders (plural) comprise at least two members (plural) that are opposed to each other. This further makes it unclear what the scope is, as the structure of the holder is not clearly defined. It reads as if there are a plurality of members in a plurality of holders in a singular end effector, but it is not clear what a “holder” actually is. The Claims 1, 5, and 9 are examples of these ambiguities, but there are other examples throughout the claim set.
The claims recite both singular and plural holder(s), without clarifying what the structure of a holder is, which further makes it unclear what the scope of “opening width” is relative to the structure of the holder(s). Due to these ambiguities, the scope of the claims is not able to be ascertained, and the Claims 1-14 are indefinite.
Amended Claim 3 recites: “a first area comprising another object area and an rear side area, an object other than the holding object existing in the another object area, the rear side area being spread to an opposite side of the holding object area from the another object area”. This claim language is still indefinite because it is not clear what the frame of reference is for the terms “rear”, “spread”, or “opposite side”. Is the rear side relative to the holding object’s orientation? The robot’s approach direction? Or something else? The phrase “spread to an opposite side” is also unclear because it is not understood what the term ‘spread’ is intended to entail in this context. Furthermore, it is unclear what the ‘opposite side’ from the holding object area from the another area is. What is the frame of reference? The language of this claim discusses different positional relationships without a clear frame of reference. In addition to this, the claim is still written narratively which makes it difficult to understand/indefinite.
Amended Claim 6 recites “…wherein the controller is configured to calculate a first center position which is a center position of the possible existence area for the holder or each of the holders of the end effector…”. This is still indefinite with respect to the ambiguity of the scope of holder(s), but also because it is unclear what the controller is calculating “a center position of the possible existence area” (singular center position) “for each of the holders” (plural holders) or for “the holder” (singular). Is this the position between all of the holders, or what would this entail for a singular holder? Are there multiple existence areas for each holder?
Amended Claim 7 recites: “assume an intersecting point between a moving straight line and a perpendicular line as the first center position” which defines the first center position as an intersection of a moving straight line and a perpendicular line. Claim 7 then recites “the perpendicular line being drawn from the first center position to the moving straight line” which requires the first center position to already be known because otherwise it is unclear how the perpendicular line could be “drawn from” the first center position that has not been yet been defined/determined.
Additionally, the claim states that the first center point is the intersecting point between a moving straight line and a perpendicular line, which indicates that the point lies on the straight line (intersection). However, the claim then recites that this first center point is assumed in response to the first center point being deviated from the moving straight line (the point deviated/not on the line). This appears to be a spatial contradiction and it is unclear if the claim is reciting that the first center position is both on and deviated off of the moving straight line at the same time.
Additionally, the term “moving straight line” is indefinite because it is unclear what the straight line is moving relative to (frame of reference). The phrase “opening-closing direction of the holder or holders” is also indefinite because of the ambiguity with respect to the terms “holder” or “holders”. Which holder(s) is the opening-closing direction relative to? (frame of reference). Its not clear what the holder is, and what it would look like if there are multiple holders, which makes it difficult to determine what an opening-closing direction would be (for example, if there are three claws or fingers such as illustrated in the instant applications Figures 6B/6C, what is the direction?)
Amended Claim 8 recites: “the center position of the end effector area” which is different than the “center position of the possible existence area” recited in Claim 6 from which 8 depends on. This lacks antecedent basis and is unclear.
Furthermore, Claim 8 recites “the part of the possible existence area is outside the first center position in a direction parallel to the moving straight line.” A position is a point in space, and thus it is unclear how the possible existence area is “outside” the first center position. Is this intended to mean that the possible existence area does not include the first center position? Being outside of a point is not defined and it is unclear what this entails spatially, and thus renders this claim indefinite in addition to the other reasons stated above.
Amended Claim 11 recites “calculate a first appropriateness value representing appropriateness of the plurality of holding positions…”. It is unclear if the first appropriateness value (singular value) is intended to represent the plurality of holding positions (plural) or if it is intended to be a (singular) appropriateness value per each of the plurality of holding positions.
Additionally, Claim 11 recites “the holding position” being estimated based on the “first” appropriateness value, after previously reciting a plurality of holding positions. Which holding position is being estimated if the first appropriateness value is representing the appropriateness of a plurality of holding positions?
Claim 13 recites: "calculate a second appropriateness value based on the first appropriateness values”. This lacks antecedent basis because Claim 11 from which 13 depends on only claims a singular “first appropriate value”, so it is not clear where the (plural) “first appropriate values” comes from. Furthermore, it is unclear what holding position based on the second appropriateness value is being estimated with respect to “estimate the holding position based on the second appropriateness value”. Thus, this claim is also indefinite for these reasons in addition to those discussed above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Re Amended Claim 14 recites: A holding parameter estimation method comprising: Acquiring end effector information concerning an end effector including a holder or holders configured to hold a holding object, holding target information indicating the holding object, and depth data about the holding object; acquiring an end effector model based on the end effector information, the end effector model indicating an end effector area in which the end effector can exist; and estimating an opening width of the holder or holders for the holding of the holding object, the opening with estimated based on the end effector model, the holding target information, and the depth data.
Under Step 1: Claim 14 is a method.
Under Step 2A Prong 1: The claim recites an judicial exception of abstract idea of mental
processes. The additional elements are crossed out.
A person can utilize received data/information to mentally or via pen and paper determine an end effector model which indicates an area that the end effector can exist, and estimate an opening width of the holder.
Under Step 2A Prong 2, the crossed-out items in the claims are additional elements. The recited
"acquirer" is merely gathering/ "acquiring" data. The data being used to perform the abstract idea
generically using computer part(s). See MPEP § 2106.05(f, g). Thus, the additional elements do not add more to make the abstract idea into a practical application.
Under Step 2B, the additional elements are the same as Step 2A Prong 2. For the same reasons, the additional elements also are not sufficient to amount to significantly more than the abstract idea.
The examiner recommends adding a line that recites actively controlling to grab/grasp an object to overcome the 101, as long as there is sufficient support in the specification, such as amended Claim 1.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5, 9-11, and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS).
Regarding Claim 1, Nishina discloses:
A holding parameter estimation device comprising: an acquirer configured to acquire end effector information concerning an end effector including a holder or holders configured to hold a holding object with an arbitrary opening width, (See at least Figure 1 and Figure 2 which illustrates different parts of the information processing apparatus 14, which is connected to three-dimensional sensor 16, and includes an "interface" as per ¶0062. Additionally, information processing apparatus 14 incudes a storage medium 20 which "stor[es] hand shape data HD of the multi-fingered hand H")
holding target information indicating the holding object, and depth data about the holding object; (See at least ¶0059 via "A three-dimensional measurement sensor 16 (an example of the “three-dimensional measurement sensor”) captures an image of an area that contains a workpiece W (an example of the “object”), which is an object to be gripped, and outputs, for each image element (also referred to as “pixel”), range data representing the distance between the three-dimensional measurement sensor 16 and the object present in the area" *wherein the range data is the depth data and the holding target information indicating the holding object is illustrated as "the" object/"workpiece W" in which the range is determined for. Additionally, see ¶0062 via "workpiece shape data")
and a controller configured to acquire an end effector model based on the end effector information, the end effector model indicating an end effector area in which the end effector can exist, (See at least Figure 2 via "Robot control unit 12" which is illustrated to receive the data, such as the position/posture of the holder/Hand H-which is a model indicating an area which the end effector can exist: ¶0066 via "The gripping position/posture recognition unit 22 recognizes the gripping position/posture at which the fingers of the multi-fingered hand H are in contact with and grips a workpiece W based on the range image data DD, and the hand shape data HD and the workpiece shape data WD that are read out from the storage medium 20.". Additionally, for clarification regarding the area in which the end effector can exist, see at least: ¶0067-¶0069, ¶0080 via "it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture.", and also ¶0090-¶0098)
estimate an opening width of the holder or holders for the holding of the holding object, the opening width estimated based on the end effector model, the holding target information, and the depth data, (See at least ¶0068 via "The hand opening width update unit 26 updates the set value of the opening width of the multi-fingered hand H based on the range image data DD interpolated by the range image interpolation unit 24" and ¶0069 via "The optimum gripping position/posture calculation unit 28 calculates, as an optimum gripping position/posture, the gripping position/posture set by the gripping position/posture recognition unit 22 and the opening width of the multi-fingered hand H set by the hand opening width update unit 26.")
PNG
media_image1.png
543
720
media_image1.png
Greyscale
and control the end effector based on the opening width of the holder or holders (See at least ¶0103 via "The robot control unit 12 outputs a control instruction to instruct the robot R and the multi-fingered hand H to take the input optimum gripping position/posture").
Regarding Claim 2, Nishina discloses the holding parameter estimation device according to Claim 1.
Furthermore, Nishina discloses: wherein the controller is configured to create an approach map based on the holding target information and the depth data, the approach map indicating an approach area in which the end effector can be opened without interference with an object other than the holding object at a height at which the holding object is held and (See at least ¶0080 via "Then, the gripping position/posture recognition unit 22 calculates the gripping positions/postures of the fingers of the multi-fingered hand H for getting in contact with and gripping a workpiece W, based on the information relating to the positions/postures of the workpieces W extracted in step S33, and the grip-position information and the hand shape data HD that have been read out from the gripping position database DB. Here, it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture." and also ¶0081 via " Then, the range image interpolation unit 24 interpolates the unmeasured areas UA using range data in order to perform determination of interference in the next step S36 (step S35)." also see Figures 5A-5C which shows the calculation of the range/distance, and corresponds to a height of a workpiece(s), and also Figures 7A and 7B which shows the consideration of the range data which is the height.)
estimate the opening width based on the approach area and the end effector area (See at least ¶0088 via "Then, the hand opening width update unit 26 updates the opening width (opening degree) of the multi-fingered hand H (an example of “distance between a plurality of fingers”) based on the range image data DD in which the range data of the unmeasured areas UA is interpolated (step S36)").
Regarding Claim 3, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the controller is configured to create a holding object area where the holding object exists, (See at least Figure 4 which illustrates a holding object: workpiece W, which has boundaries as shown and is thus the holding object area)
a first area comprising another object area and an rear side area, an object other than the holding object existing in the another object area, the rear side area being spread to an opposite side of the holding object area from the another object area, and (See at least Figure 4 which illustrates other objects that are not the holding object, and are defined by boundaries as shown, and are thus considered another area which is adjacent to the holding object area) *Note the 112(b) rejection
PNG
media_image2.png
418
532
media_image2.png
Greyscale
a second area where the end effector can be opened around the holding object without interference with the holding object, and (See at least ¶0104 via "In accordance with the control instruction, the robot R and the multi-fingered hand H approach the workpiece W with the opening width of the multi-fingered hand H at the optimum gripping position/posture selected in step S37, and take the optimum gripping position/posture. Then, the robot R and the multi-fingered hand H reduce the distance between the fingers of the multi-fingered hand H based on the control instruction output from the robot control unit 12, and grip the workpiece W." *Wherein the second area is the area where the holder can be opened without interference, and is thus the area the holder is opened in when travelling/approaching to grasp/hold the object, which has already been determined to have no interference)
create the approach map by excluding the holding object area and the first area from the second area (Wherein the second area is the area in which the holder can open and travel toward the object to be held, and is determined to have no interference from the object (holding object area) or the first area (another object), and is thus the approach map is an area that the end effector can be opened without interference and thus excludes the areas that would have interference (holding object area and first area)).
Regarding Claim 5, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the controller is configured to estimate a possible existence area in which the approach area is overlapped with the end effector model for the holder or each of the holders of the end effector, and (See at least ¶0107 via "In the case of the comparative example, as a result of interpolating the unmeasured area UA, the width of the workpiece W is interpolated to be smaller than the actual width of the workpiece W. Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped." and also ¶0108. *Wherein the device mitigates the damage by increasing the width of the holder when there is unmeasured areas where the approach area might overlap.) *Note: see 112(b) rejection.
estimate a first arbitrary point in the possible existence area of the holder or each of the holders as a position of the holder or holders (See at least ¶0095 via "Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H.").
Regarding Claim 9, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the end effector comprises holders, and the holders comprises at least two members that are opposed to each other and that are movable (See at least Figure 1 and also ¶0050 via "Note that the number of fingers of the multi-fingered hand H is not limited as long as the multi-fingered hand H has a plurality of fingers. For example, the multi-fingered hand H may have three fingers or more.") *Note the 112b rejection, as it is unclear what the scope of holder is and thus what the scope of the end effector is and how it has (multiple) holders.
Regarding Claim 10, Nishina discloses the holding parameter estimation device according to Claim 9.
Furthermore, Nishina discloses: wherein the controller is configured to create an opening width model in which the end effector can exist within a range of the opening width, (See at least ¶0090 via "First, it is determined whether or not the current opening width (an example of a “first distance”) of the multi-fingered hand H set in step S34 is equal to or smaller than the maximum value of the maximum opening width that the multi-fingered hand H can have (sub step S61)." *wherein the maximum opening width is the opening width model)
create at least one rule map including a map defining a holding position of the holding object based on the holding target information, the holding position used by the end effector for the holding, and (See at least Figure 4 which illustrates the target workpiece(s) and the closer (in range) objects being brighter: ¶0076 via "FIG. 4 is an example showing range image data obtained by capturing an image of a plurality of workpieces W (fruits, oranges) piled up in bulk as a two-dimensional image in which the workpieces W are brighter, the closer they are to a viewer. I
estimate a contact position where the end effector is in contact with the holding object as the holding position, the contact position estimated based on the opening width model and the at least one rule map (See at least ¶0079 via "The gripping position/posture of the multi-fingered hand H for gripping a workpiece W is calculated as will be described below (step S34)…").
Regarding Claim 11, Nishina discloses the holding parameter estimation device according to Claim 10.
Furthermore, Nishina discloses: wherein the controller is configured to estimate a plurality of holding positions, calculate a first appropriateness value representing appropriateness of the plurality of holding positions based on the opening width model and the at least one rule map, and estimate the holding position based on the first appropriateness value (See at least ¶0079 via " The grip-position information may include a plurality of grippable positions with respect to one workpiece W. Furthermore, score information associated with each grippable position may also be included").
Regarding Claim 14, Nishina discloses:
A holding parameter estimation method comprising: (See at least Figure 3 regarding method)
acquiring end effector information concerning an end effector including a holder or holders configured to hold a holding object, (See at least Figure 3 via "HD" which is hand shape data, also ¶0080 via "the gripping position/posture recognition unit 22 reads out the hand shape data HD from the storage medium 20").
holding target information indicating the holding object, and depth data about the holding object; (See at least ¶0059 via "A three-dimensional measurement sensor 16 (an example of the “three-dimensional measurement sensor”) captures an image of an area that contains a workpiece W (an example of the “object”), which is an object to be gripped, and outputs, for each image element (also referred to as “pixel”), range data representing the distance between the three-dimensional measurement sensor 16 and the object present in the area" *wherein the range data is the depth data and the holding target information indicating the holding object is illustrated as "the" object/"workpiece W" in which the range is determined for. Additionally, see ¶0062 via "workpiece shape data")
acquiring an end effector model based on the end effector information, the end effector model indicating an end effector area in which the end effector can exist; and (See at least Figure 3 via S34 via "calculate gripping position/posture" and also ¶0066 via "The gripping position/posture recognition unit 22 recognizes the gripping position/posture at which the fingers of the multi-fingered hand H are in contact with and grips a workpiece W based on the range image data DD, and the hand shape data HD and the workpiece shape data WD that are read out from the storage medium 20." *Wherein the position/posture of the holder/Hand H is a model indicating an area which the end effector can exists. Additionally, for clarification regarding the area in which the end effector can exist, see at least: ¶0067-¶0069, ¶0080 via "it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture.", and also ¶0090-¶0098)
estimating an opening width of the holder or holders for the holding of the holding object, the opening with estimated based on the end effector model, the holding target information, and the depth data (See at least Figure 3 via S36 via "update opening width of multi-fingered hand H based on interpolated range image data DD").
PNG
media_image3.png
731
547
media_image3.png
Greyscale
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS).
Regarding Claim 4, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the controller is configured to create the approach map by excluding expansion areas from the second area, the expansion areas which are areas expanded from each of (See at least Figures 7A-7B which illustrate the holder increasing the opening width when there is an unmeasured area UA- which can be interpolated to determine an unknown edge of a workpiece. The expanding/increasing width of the holder illustrates the space that is expanded from the holding object area and is thus excluded from the second area. Also see at least ¶0108 via "In the case of one or more embodiments, after the gripping position/posture of the multi-fingered hand H when gripping the workpiece W has been decided, the entire ranges, along the axis in the direction in which the multi-fingered hand H is opened and closed, of the unmeasured areas UA are interpolated using the closer (smaller) one of adjacent distances represented by the range data, and the opening width of the multi-fingered hand H is set so that the multi-fingered hand H does not interfere with the workpiece W or the like even based on the interpolated range image data DD. Then, by advancing toward the workpiece W while keeping an opening width that is equal to or larger than this opening width at least in the vicinity of the workpiece W, and then reducing the distance between the fingers of the hand H, the multi-fingered hand H can grip the workpiece W without damaging it.")
PNG
media_image4.png
659
555
media_image4.png
Greyscale
However, Nishina does not explicitly disclose the expansion areas which are areas expanded from each of the first area.
Nevertheless, it would have been obvious to one of ordinary skill in the art to consider the other objects/first area as obstacles and include an expansion area as a buffer zone to avoid the holder colliding with a nearby object: "…Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped…" [Nishina ¶0107] in order to avoid damaging an object or the operation process being stopped. Furthermore, since the unmeasured areas can exist near other objects, it would have been obvious to consider an opening width that expands outside of the unmeasured areas of both the object areas and the 'first' area (other objects).
Regarding Claim 6, Nishina discloses the holding parameter estimation device according to Claim 5.
Furthermore, Nishina discloses: wherein the holder or holders are movable in a certain direction, and (See at least Figures 7A-7B which show the holder moving in a certain direction)
wherein the controller is configured to calculate a first center position which is a center position of the possible existence area for the holder or each of the holders of the end effector, and (See at least ¶0095 via "Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H." *Wherein the coordinates calculated of each tip of each finger is the center position of "each holder"--See 112(b) rejections).
estimate a second arbitrary position as the position of the holder or holders, the second arbitrary position at which the holder or holders do not interfere with the holding object and (See at least ¶0096 via "Thus, interference of the multi-fingered hand H is determined using the calculated finger areas of the multi-fingered hand H and the interpolated range image data DD (sub step S62). For example, by determining whether or not, on the range image data DD, the finger areas of the multi-fingered hand H overlap an area that is likely to interfere with the multi-fingered hand H (for example, an area whose range data has a value equal to or smaller than a predetermined value), the likelihood of interference of the multi-fingered hand H is determined.").
(See at least ¶0081 via "Then, the range image interpolation unit 24 interpolates the unmeasured areas UA using range data in order to perform determination of interference in the next step S36 (step S35)." and Figures 5A-5C, wherein the system is determining the optimal width of the holder (center position of each finger/holder) in order to avoid interference when there is an unmeasured area).
However, Nishina does not explicitly disclose another object being in the area passed: an object other than the holding object Nevertheless, it would have been obvious to one of ordinary skill in the art to consider other objects in a traversed area as obstacles in order to mitigate interference by avoiding the holder colliding with a nearby object: "…Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped…" which would avoid damaging an object or the operation process being stopped.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS) in view of Hoffman et. al. (US 20220234208 A1).
Regarding Claim 7, Nishina discloses the holding parameter estimation device according to Claim 6.
However, Nishina does not explicitly disclose, but Hoffman—who is directed towards image based guidance for picking—discloses: wherein, the controller is configured to assume an intersecting point between a moving straight and a perpendicular line as the first center position in response to the first center position being deviated from the moving straight line, the moving straight line passing through a center position of the end effector area in an opening-closing direction of the holder or holders, the perpendicular line being drawn from the first center position to the moving straight line, and estimate the position of the holder or holders corresponding to the opening width (See at least ¶0019 via "… (c) calculating a deviation of an orientation of the wire gripper from being parallel to the second portion of the wire based on the camera images taken in step (a); (d) controlling the robot to rotate the wire gripper so that the orientation of the wire gripper matches the orientation of the second portion of the wire…" and "…(k) determining that the distance calculated in step (j) is less than the first threshold; and (l) controlling a pair of gripper fingers of the wire gripper to move to respective positions where the gripper fingers constrain displacement of the second portion of the wire in directions perpendicular to an axis of the wire in response to step (k)…". Additionally see ¶0059. *Wherein, Hoffman is using image guidance to determine and correct a deviation of the gripper/holder---See 112(b) rejections).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Hoffman in order to correct for deviations detected by the cameras, as "Capturing images having multiple viewpoints, such as using two or more cameras, enables accurate positioning of the tip 9 of wire gripper 30 relative to wire 11 and then relative to wire contact 3." [Hoffman ¶0055] and so the target item (in this case, a wire) can properly be gripped: "Given the estimated deviation in orientation, robot movement commands are executed to rotate the wire gripper 30 until its orientation matches the orientation of wire 11 (step 108). The orientations match if the axes of the wire gripper 30 and wire 11 are parallel or at a sufficiently small angle that wire 11 may enter the space between gripper fingers 52a and 52b when open wire gripper 30 is lowered." [Hoffman ¶0058].
Regarding Claim 8, Nishina discloses the holding parameter estimation device according to Claim 6.
However, Nishina does not explicitly disclose, but Hoffman—who is directed towards image based guidance for picking—discloses: wherein, the controller is configured to search for the approach area in a part of the possible existence area in response to the approach area not existing upon displacement of the holder or holders from the first center position toward the center position of the end effector area, the part of the possible existence area is outside the first center position in a direction parallel to the moving straight line (See at least ¶0019 via "… (c) calculating a deviation of an orientation of the wire gripper from being parallel to the second portion of the wire based on the camera images taken in step (a); (d) controlling the robot to rotate the wire gripper so that the orientation of the wire gripper matches the orientation of the second portion of the wire…" and "…(k) determining that the distance calculated in step (j) is less than the first threshold; and (l) controlling a pair of gripper fingers of the wire gripper to move to respective positions where the gripper fingers constrain displacement of the second portion of the wire in directions perpendicular to an axis of the wire in response to step (k)…". *Wherein the deviated approach is unavailable because it would not result in picking the item, and so the searching for the new approach is the approach/realignment and correction to be able to accurately pick the item ). *See 112b rejection
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Hoffman in order to correct for deviations so the target item (in this case, a wire) can properly be gripped and the process can continue: "Given the estimated deviation in orientation, robot movement commands are executed to rotate the wire gripper 30 until its orientation matches the orientation of wire 11 (step 108). The orientations match if the axes of the wire gripper 30 and wire 11 are parallel or at a sufficiently small angle that wire 11 may enter the space between gripper fingers 52a and 52b when open wire gripper 30 is lowered." [Hoffman ¶0058].
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS) in view of Kusano et. al. (US 20200094406 A1 -- Corresponding to US11034018B2 of IDS).
Regarding Claim 12, Nishina discloses the holding parameter estimation device according to Claim 11.
However, Nishina does not explicitly disclose, but Kusano--who is directed towards a grasping system--discloses: wherein the opening width model and the at least one rule map are represented by values that are allocated to respective positions and that indicate the appropriateness as the positions where the holding object is held, and (See at least ¶0083 via "Next, the estimation part 106 may select information with a high score from the data of the depth and posture of the graspable gripper 16 output by the learning model and output the information through the output part 108 (step S22). For example, the output maps of the PLM and the PCM in themselves are referred to as the score.")
wherein the controller is configured to calculate the first appropriateness values by computing the values allocated to respective positions in response to the opening width model being superimposed on the at least one rule map (See at least ¶0082 via "Note that the outputs of the PCM and the PLM may be output as images as illustrated in FIG. 4 through the output part 108. The image of the PLM may output the locations of the gripper 16 which have a high possibility that the gripper 16 is able to grasp the target object as an aggregation of dots." and also ¶0052 via "For example, each pixel of the PLM is a map which shows a value near one when the target object can be grasped if the gripper 16 exists at the location (x, y) corresponding to the pixel, and shows a value near 0.5 or a value of 0.5 or less when the target object cannot be grasped. As described later, the output data of the PLM and the PCM may become a region of (0, 1) through a sigmoid function.")
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Kusano's maps and scores in order to improve the selection of graspable locations/postures and make it possible to utilize a higher-dimensional DOF which would allow for more mobility: "it is possible to estimate information representing a graspable state for higher-dimensional degree-of-freedom by using the supervised data based on other parameters such as a bending state of the claw of the gripper 16 at a joint, and a distance between the claws." [Kusano ¶0089].
Regarding Claim 13, Nishana discloses the holding parameter estimation device according to Claim 11.
However, Nishana does not explicitly disclose, but Kusano--who is directed towards a grasping system--discloses: wherein the at least one rule map includes a plurality of maps, wherein the controller is configured to calculate a second appropriateness value based on the first appropriateness values based on the maps included in the plurality of rule maps, respectively, and a map coefficient defined for each of the maps, and to estimate the holding position based on the second appropriateness value (See at least ¶0051 via " In this learning model, when three pieces of images related to RGB image with 200×200 pixels and one piece of depth map image are input as input images, a location map (PLM: predicted location map) and a posture and depth map (PCM: predicted configuration map) which are predicted to be able to be grasped may be output. The PLM and the PCM may be information indicating whether an object can be grasped by each pixel photographed by the camera 18." and ¶0052 via "For example, each pixel of the PLM is a map which shows a value near one when the target object can be grasped if the gripper 16 exists at the location (x, y) corresponding to the pixel, and shows a value near 0.5 or a value of 0.5 or less when the target object cannot be grasped" and ¶0083 via "Next, the estimation part 106 may select information with a high score from the data of the depth and posture of the graspable gripper 16 output by the learning model and output the information through the output part 108 (step S22). For example, the output maps of the PLM and the PCM in themselves are referred to as the score.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Kusano's multiple maps and appropriateness scores in order to improve the selection of graspable locations/postures and make it possible to utilize a higher-dimensional DOF which would allow for more mobility: "it is possible to estimate information representing a graspable state for higher-dimensional degree-of-freedom by using the supervised data based on other parameters such as a bending state of the claw of the gripper 16 at a joint, and a distance between the claws." [Kusano ¶0089].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAYLA RENEE DOROS whose telephone number is (703)756-1415. The examiner can normally be reached Generally: M-F (8-5) EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.D./Examiner, Art Unit 3657
/ABBY LIN/Supervisory Patent Examiner, Art Unit 3657