DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This non-final office action is a response to the RCE received on 04/27/2026. Claims 1-14 are pending. Claims 1, 3-9, 11, and 13-14 have been amended.
Response to Arguments
Applicant’s amendments overcome the claim objections of Claims 3 and 9.
Applicant’s amendments overcome the 101 rejection of Claim 14.
Applicant’s amendments overcome the 112(b) rejection of Claims 1-14 regarding the indefiniteness/ambiguity of the term “holder”. Applicant’s amendments also overcome the previous 112(b) rejections of Claim 3, 6, 11, and 13.
Applicant’s amendments overcome some of the previous 112(b) rejections for Claims 7-8, but also introduce new 112(b) issues (See below).
Applicant’s additional arguments with respect to the claims have been considered but are not persuasive with respect to the broadest reasonable interpretation of the claims. Applicant asserts that “the end effector model is [required] to be map-like spatial data that can be overlaid on other maps.”, and further asserts that the prior art, Nishina, does not disclose the claim limitation “the end effector model indicating an end effector area within a map in which the end effector can exist, " On Page 11 of the remarks. The claim language does not require such a narrow interpretation of the ‘map’. The claimed map does not require a particular format, structure, or specific type of spatial representation. Furthermore, the claim language does not require the end effector model to be a map itself, instead the language of “end effector model indicating an end effector area within a map” is broader, and can encompass the end effector/its occupied area (area which it can exist) to be spatially represented in an area that is mapped (such as an image that has a defined coordinate system).
Therefore, under broadest reasonable interpretation, the claimed map does include Nishina’s image/coordinate (corresponding to map) calculated representation of the areas of occupied space of the end effector’s fingers (corresponding to an area in which the end effector can exist). See at least Nishina ¶0095 via “Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H.” as well as ¶0090-¶0098.
Furthermore, the claim language is not so narrow to require the end effector area in which the end effector can exist to include every possible positioning/area that the end effector is able to be. Additionally, the specific terminology of “model” or “map” is not required, and the terms are broad enough that they encompass Nishina’s hand-shape data and the image with a defined coordinate space/representations.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Amended Claim 7 recites: “wherein, the controller is configured to assume an intersecting point between a virtual straight line and a perpendicular line as an adjusted first center position in response to a calculated first center position being deviated from the virtual straight line or intersecting points between virtual straight lines and perpendicular lines as the center positions in response to the first center positions being deviated from the virtual straight lines,
the virtual straight line passing through a center position of the end effector area in an opening-closing direction of the at least one holder, the virtual straight lines passing through center positions of the end effector area in opening-closing directions of the at least one holder,
the perpendicular line being drawn from the first center position to the virtual straight line, the perpendicular lines being drawn from the first center positions to the respective virtual straight lines, and estimate the position of the at least one holder corresponding to the opening width.”
It is unclear what the relationship between “first center position” and “first center positions” entails. The language first recites “a calculated first center position” and further recites “the first center positions”. It is unclear how the singular first center position corresponds to a plurality of first center positions. Additionally, the language states “intersecting points between virtual straight lines and perpendicular lines as the center positions” but does not make clear whether the center positions are the same or different than the “first center positions” which makes the language indefinite.
Furthermore, the relationship between the virtual straight lines, perpendicular lines, intersecting points, and first center positions is not clearly written or understood. The claim first recites a singular first center position but proceeds to recite plural virtual straight lines, intersecting points, perpendicular lines, and first center positions, but does not clearly define what the initial first center position corresponds to (in terms of which of the virtual lines, perpendicular lines, etc). The claim recites “respective” virtual straight lines, but it is still unclear what the relationship is in terms of whether there are intended to be multiple holders (one or more). The spatial relationship of the lines, points, and positions is unclear and indefinite.
It is unclear whether the “intersecting point” and “intersecting points” are intended to be the same type of point/position, because the claim recites “an intersecting point between a virtual straight line and a perpendicular line as an adjusted first center position” but then recites “intersecting points between virtual straight lines as the center positions”. It is unclear whether the center positions (plural) are multiple adjusted first center positions or if they are separate center positions, thus the claim language is indefinite.
Amended Claim 8 recites: “wherein, the controller is configured to search for the approach area in a part of the possible existence area in response to the approach area not existing upon displacement of the at least one holder from the first center position toward the center position of the end effector area or approach areas in a part of the possible existence areas in response to the approach areas not existing upon displacement of the respective holders from the first center position toward the center position of the end effector area,
the part of the possible existence area being outside the first center position in a direction parallel to the virtual straight line, the part of the possible existence areas being outside the first center positions in a direction parallel to the virtual straight lines.”
Claim 8 was previously rejected for reciting similar language to the amended limitation: “the part of the possible existence area being outside the first center position in a direction parallel to the virtual straight line.”. The amendments do not clear up the ambiguity. It is still unclear, given that a position is a point in space, how the possible existence area is “outside” the first center position. Is this intended to mean that the possible existence area is any area that does not include the first center position? Being outside of a point, even with the addition of being in a direction parallel to the virtual straight line, is not clearly defined and it is unclear what this entails spatially.
Furthermore, introducing the plural alternative introduces the same problem multiple times. This also furthers the ambiguity introduces by amended Claim 7 from which amended Claim 8 depends on: Is each area associated with one respective first center position and virtual straight line? It is unclear which of the first center positions is used to define which of the possible existence areas or which of the virtual straight lines is used to establish the recited “direction parallel to the virtual straight line”. These ambiguities render the claim language indefinite.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 5, 9-11, and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS).
Regarding Claim 1, Nishina discloses:
A holding parameter estimation device comprising: an acquirer configured to acquire end effector information concerning an end effector including at least one holder configured to hold a holding object with an arbitrary opening width, (See at least Figure 1 and Figure 2 which illustrates different parts of the information processing apparatus 14, which is connected to three-dimensional sensor 16, and includes an "interface" as per ¶0062. Additionally, information processing apparatus 14 incudes a storage medium 20 which "stor[es] hand shape data HD of the multi-fingered hand H" )
holding target information indicating the holding object, and depth data about the holding object; and (See at least ¶0059 via "A three-dimensional measurement sensor 16 (an example of the “three-dimensional measurement sensor”) captures an image of an area that contains a workpiece W (an example of the “object”), which is an object to be gripped, and outputs, for each image element (also referred to as “pixel”), range data representing the distance between the three-dimensional measurement sensor 16 and the object present in the area" *wherein the range data is the depth data and the holding target information indicating the holding object is illustrated as "the" object/"workpiece W" in which the range is determined for. Additionally, see ¶0062 via "workpiece shape data")
a controller configured to acquire an end effector model based on the end effector information, the end effector model indicating an end effector area within a map in which the end effector can exist, (See at least Figure 2 via "Robot control unit 12" which is illustrated to receive the data, such as the position/posture of the holder/Hand H-which is a model indicating an area which the end effector can exist: ¶0066 via "The gripping position/posture recognition unit 22 recognizes the gripping position/posture at which the fingers of the multi-fingered hand H are in contact with and grips a workpiece W based on the range image data DD, and the hand shape data HD and the workpiece shape data WD that are read out from the storage medium 20". Additionally, Nishina discloses the area in which the end effector can exist, see at least: ¶0067-¶0069, ¶0080 via "it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture.". Also see ¶0090-¶0098, and specifically see ¶0095 via “Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H.” and ¶0096 via "Thus, interference of the multi-fingered hand H is determined using the calculated finger areas of the multi-fingered hand H and the interpolated range image data DD (sub step S62)." **Which under BRI, corresponds to the end effector area within a map in which the end effector can exist)
estimate an opening width of the at least one holder for the holding of the holding object, the opening width estimated based on the end effector model, the holding target information, and the depth data, and (See at least ¶0068 via "The hand opening width update unit 26 updates the set value of the opening width of the multi-fingered hand H based on the range image data DD interpolated by the range image interpolation unit 24" and ¶0069 via "The optimum gripping position/posture calculation unit 28 calculates, as an optimum gripping position/posture, the gripping position/posture set by the gripping position/posture recognition unit 22 and the opening width of the multi-fingered hand H set by the hand opening width update unit 26.")
PNG
media_image1.png
543
720
media_image1.png
Greyscale
control the end effector based on the opening width of the at least one holder (See at least ¶0103 via "The robot control unit 12 outputs a control instruction to instruct the robot R and the multi-fingered hand H to take the input optimum gripping position/posture").
Regarding Claim 2, Nishina discloses the holding parameter estimation device according to Claim 1.
Furthermore, Nishina discloses: wherein the controller is configured to create an approach map based on the holding target information and the depth data, the approach map indicating an approach area in which the end effector can be opened without interference with an object other than the holding object at a height at which the holding object is held and (See at least ¶0080 via "Then, the gripping position/posture recognition unit 22 calculates the gripping positions/postures of the fingers of the multi-fingered hand H for getting in contact with and gripping a workpiece W, based on the information relating to the positions/postures of the workpieces W extracted in step S33, and the grip-position information and the hand shape data HD that have been read out from the gripping position database DB. Here, it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture." and also ¶0081 via " Then, the range image interpolation unit 24 interpolates the unmeasured areas UA using range data in order to perform determination of interference in the next step S36 (step S35)." also see Figures 5A-5C which shows the calculation of the range/distance, and corresponds to a height of a workpiece(s), and also Figures 7A and 7B which shows the consideration of the range data which is the height.)
Regarding Claim 5, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the controller is configured to estimate a possible existence area in which the approach area is overlapped with the end effector model for the at least one holder of the end effector, and (See at least ¶0107 via "In the case of the comparative example, as a result of interpolating the unmeasured area UA, the width of the workpiece W is interpolated to be smaller than the actual width of the workpiece W. Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped." and also ¶0108. *Wherein the device mitigates the damage by increasing the width of the holder when there is unmeasured areas where the approach area might overlap.)
estimate a first arbitrary point in the possible existence area of the at least one holder as a position of the at least one holder (See at least ¶0095 via "Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H.").
Regarding Claim 9, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the end effector comprises at least two members that are opposed to each other and that are movable as the at least one holder (See at least Figure 1 and also ¶0050 via "Note that the number of fingers of the multi-fingered hand H is not limited as long as the multi-fingered hand H has a plurality of fingers. For example, the multi-fingered hand H may have three fingers or more.").
Regarding Claim 10, Nishina discloses the holding parameter estimation device according to Claim 1.
Furthermore, Nishina discloses: wherein the controller is configured to create an opening width model in which the end effector can exist within a range of the opening width, (See at least ¶0090 via "First, it is determined whether or not the current opening width (an example of a “first distance”) of the multi-fingered hand H set in step S34 is equal to or smaller than the maximum value of the maximum opening width that the multi-fingered hand H can have (sub step S61)." *wherein the maximum opening width is the opening width model)
create at least one rule map including a map defining a holding position of the holding object based on the holding target information, the holding position used by the end effector for the holding, and (See at least Figure 4 which illustrates the target workpiece(s) and the closer (in range) objects being brighter: ¶0076 via "FIG. 4 is an example showing range image data obtained by capturing an image of a plurality of workpieces W (fruits, oranges) piled up in bulk as a two-dimensional image in which the workpieces W are brighter, the closer they are to a viewer.)
estimate a contact position where the end effector is in contact with the holding object as the holding position, the contact position estimated based on the opening width model and the at least one rule map (See at least ¶0079 via "The gripping position/posture of the multi-fingered hand H for gripping a workpiece W is calculated as will be described below (step S34)…" and ¶0076).
Regarding Claim 11, Nishina discloses the holding parameter estimation device according to Claim 10.
Furthermore, Nishina discloses: wherein the controller is configured to estimate a plurality of holding positions, calculate first appropriateness values representing appropriateness of the plurality of candidate holding positions, respectively, based on the opening width model and the at least one rule map, and estimate the holding position based on the first appropriateness values (See at least ¶0079 via " The grip-position information may include a plurality of grippable positions with respect to one workpiece W. Furthermore, score information associated with each grippable position may also be included" and ¶0076).
Regarding Claim 14, Nishina discloses:
A holding parameter estimation method comprising: (See at least Figure 3 regarding method)
(Regarding the method steps, See Claim 1 rejection which discloses the device that comprises an acquirer and controller configured to do the same steps).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS) in view of Oka et. al. (US 20200391385 A1).
Regarding Claim 3, Nishina discloses the holding parameter estimation device according to Claim 2.
Furthermore, Nishina discloses: wherein the controller is configured to create a holding object area where the holding object exists, (See at least Figure 4 which illustrates a holding object: workpiece W, which has boundaries as shown and is thus the holding object area. Also see ¶0045 and ¶0135)
create a first area and a second area where the end effector can be opened without interference with the holding object, (See at least ¶0104 via "In accordance with the control instruction, the robot R and the multi-fingered hand H approach the workpiece W with the opening width of the multi-fingered hand H at the optimum gripping position/posture selected in step S37, and take the optimum gripping position/posture. Then, the robot R and the multi-fingered hand H reduce the distance between the fingers of the multi-fingered hand H based on the control instruction output from the robot control unit 12, and grip the workpiece W." *Wherein the second area is the area where the holder can be opened without interference, and is thus the area the holder is opened in when travelling/approaching to grasp/hold the object, which has already been determined to have no interference. Also see ¶0108 via "after the gripping position/posture of the multi-fingered hand H when gripping the workpiece W has been decided, the entire ranges, along the axis in the direction in which the multi-fingered hand H is opened and closed, of the unmeasured areas UA are interpolated using the closer (smaller) one of adjacent distances represented by the range data, and the opening width of the multi-fingered hand H is set so that the multi-fingered hand H does not interfere with the workpiece W or the like even based on the interpolated range image data DD" **Wherein the area that the multi-fingered hand H does not interfere with the object is defined in Nishina, even though Nishina does not explicitly label it as a "second area".)
the first area comprising another object area where an object other than the holding object exists (See at least ¶0080 via " it is also determined whether or not any obstacle around the workpiece W or a container that accommodates the workpiece W does not interfere with the multi-fingered hand H, and the gripping positions/postures at which the interference may occur are took out from the gripping position/posture" **Wherein the area associated with an obstacle/object other than the holding object that is around the workpiece W is explicitly recognized, even though Nishina does not label it as a "first area")
create the approach map by excluding the holding object area and the first area from the second area (Wherein the second area is the area in which the holder can open and travel toward the object to be held, and is determined to have no interference from the object (holding object area) or the first area (another object), and is thus the approach map is an area that the end effector can be opened without interference and thus excludes the areas that would have interference (holding object area and first area)).
However, Nishina does not explicitly disclose the specific opposite side area spatial region separation.
Nevertheless, Oka--who is directed towards object handling—discloses: and an opposite side area extending from the another object area in a direction opposite to the holding object area with respect to the another object area, and (See at least ¶100 via "The second control region of the third region is adjacent to a side of the first control region opposite to the obstacle" and ¶0126 via "the second control region 102b is set below the second region 100 and laterally adjacent to the first control region 102a opposite to the obstacle" **Wherein the second control region is opposite to the obstacle. Further see Figure 10 and ¶0103 via "the second region 100 is set in the vicinity of the top surface of the previously set object OBJs as illustrated in the lower drawing of FIG. 10" ).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of the opposite side area such as in Oka, in order to explicitly define the spaces and areas that can be used to approach gripping a target object in order to avoid collision or unwanted interference with obstacles: "can avoid a hand or an object grasped by the hand from interfering with an obstacle, which may otherwise damage them, and efficiently transfer the object." [¶0005 Oka].
Regarding Claim 4, Modified Nishina discloses the holding parameter estimation device according to Claim 3.
Furthermore, Nishina discloses: wherein the controller is configured to create the approach map by excluding expansion areas from the second area, the expansion areas which are areas expanded from each of (See at least Figures 7A-7B which illustrate the holder increasing the opening width when there is an unmeasured area UA- which can be interpolated to determine an unknown edge of a workpiece. The expanding/increasing width of the holder illustrates the space that is expanded from the holding object area and is thus excluded from the second area. Also see at least ¶0108 via "In the case of one or more embodiments, after the gripping position/posture of the multi-fingered hand H when gripping the workpiece W has been decided, the entire ranges, along the axis in the direction in which the multi-fingered hand H is opened and closed, of the unmeasured areas UA are interpolated using the closer (smaller) one of adjacent distances represented by the range data, and the opening width of the multi-fingered hand H is set so that the multi-fingered hand H does not interfere with the workpiece W or the like even based on the interpolated range image data DD. Then, by advancing toward the workpiece W while keeping an opening width that is equal to or larger than this opening width at least in the vicinity of the workpiece W, and then reducing the distance between the fingers of the hand H, the multi-fingered hand H can grip the workpiece W without damaging it.")
PNG
media_image2.png
526
444
media_image2.png
Greyscale
However, Nishina does not explicitly disclose the expansion areas which are areas expanded from each of the first area.
Nevertheless, it would have been obvious to one of ordinary skill in the art to consider the other objects/first area as obstacles and include an expansion area as a buffer zone to avoid the holder colliding with a nearby object: "…Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped…" [Nishina ¶0107] in order to avoid damaging an object or the operation process being stopped. Furthermore, since the unmeasured areas can exist near other objects, it would have been obvious to consider an opening width that expands outside of the unmeasured areas of both the object areas and the 'first' area (other objects).
Claims 6 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS).
Regarding Claim 6, Nishina discloses the holding parameter estimation device according to Claim 5.
Furthermore, Nishina discloses: wherein the at least one holder is movable in a certain direction, and (See at least Figures 7A-7B which show the holder moving in a certain direction)
wherein the controller is configured to calculate a first center position which is a center position of the possible existence area for the at least one holder of the end effector, and (See at least ¶0095 via "Accordingly, by calculating the three-dimensional positional coordinates of the tips of the fingers of the multi-fingered hand H based on the gripping position/posture of the multi-fingered hand H calculated in step S34, and applying the formula above for each tip to calculate its position on the image coordinates, it is possible to calculate the area enclosed by the positions of the tips on the image coordinates, as the finger areas occupied by the fingers of the multi-fingered hand H." *Wherein the coordinates calculated of each tip of each finger is the center position of "each holder"--See 112(b) rejections).
estimate a second arbitrary position as the position of the at least one holder, the second arbitrary position at which the at least one holder does not interfere with the holding object and (See at least ¶0096 via "Thus, interference of the multi-fingered hand H is determined using the calculated finger areas of the multi-fingered hand H and the interpolated range image data DD (sub step S62). For example, by determining whether or not, on the range image data DD, the finger areas of the multi-fingered hand H overlap an area that is likely to interfere with the multi-fingered hand H (for example, an area whose range data has a value equal to or smaller than a predetermined value), the likelihood of interference of the multi-fingered hand H is determined.").
(See at least ¶0081 via "Then, the range image interpolation unit 24 interpolates the unmeasured areas UA using range data in order to perform determination of interference in the next step S36 (step S35)." and Figures 5A-5C, wherein the system is determining the optimal width of the holder (center position of each finger/holder) in order to avoid interference when there is an unmeasured area).
However, Nishina does not explicitly disclose another object being in the area passed: an object other than the holding object Nevertheless, it would have been obvious to one of ordinary skill in the art to consider other objects in a traversed area as obstacles in order to mitigate interference by avoiding the holder colliding with a nearby object: "…Accordingly, when the multi-fingered hand H advances toward the workpiece W, the finger L collides with the workpiece W. This may wreak enormous damage, such as the workpiece W being damaged, and the operation process being stopped…" [¶0107 Nishina] which would avoid damaging an object or the operation process being stopped.
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS) in view of Kusano et. al. (US 20200094406 A1 -- Corresponding to US11034018B2 of IDS).
Regarding Claim 12, Nishina discloses the holding parameter estimation device according to Claim 11.
However, Nishina does not explicitly disclose, but Kusano--who is directed towards a grasping system--discloses: wherein the opening width model and the at least one rule map are represented by values that are allocated to respective positions and that indicate the appropriateness as the positions where the holding object is held, and (See at least ¶0083 via "Next, the estimation part 106 may select information with a high score from the data of the depth and posture of the graspable gripper 16 output by the learning model and output the information through the output part 108 (step S22). For example, the output maps of the PLM and the PCM in themselves are referred to as the score.")
wherein the controller is configured to calculate the first appropriateness values by computing the values allocated to respective positions in response to the opening width model being superimposed on the at least one rule map (See at least ¶0082 via "Note that the outputs of the PCM and the PLM may be output as images as illustrated in FIG. 4 through the output part 108. The image of the PLM may output the locations of the gripper 16 which have a high possibility that the gripper 16 is able to grasp the target object as an aggregation of dots." and also ¶0052 via "For example, each pixel of the PLM is a map which shows a value near one when the target object can be grasped if the gripper 16 exists at the location (x, y) corresponding to the pixel, and shows a value near 0.5 or a value of 0.5 or less when the target object cannot be grasped. As described later, the output data of the PLM and the PCM may become a region of (0, 1) through a sigmoid function.").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Kusano's maps and scores in order to improve the selection of graspable locations/postures and make it possible to utilize a higher-dimensional DOF which would allow for more mobility: "it is possible to estimate information representing a graspable state for higher-dimensional degree-of-freedom by using the supervised data based on other parameters such as a bending state of the claw of the gripper 16 at a joint, and a distance between the claws." [Kusano ¶0089].
Regarding Claim 13, Nishina discloses the holding parameter estimation device according to Claim 11.
However, Nishina does not explicitly disclose, but Kusano discloses: wherein the at least one rule map includes a plurality of maps, wherein the controller is configured to calculate the first appropriateness values for the plurality of maps, respectively, wherein the controller is configured to calculate second appropriateness values based on the first appropriateness values based on the maps included in the plurality of the rule maps, respectively, and a map coefficient defined for each of the maps, and to estimate the holding position based on the second appropriateness values (See at least ¶0051 via " In this learning model, when three pieces of images related to RGB image with 200×200 pixels and one piece of depth map image are input as input images, a location map (PLM: predicted location map) and a posture and depth map (PCM: predicted configuration map) which are predicted to be able to be grasped may be output. The PLM and the PCM may be information indicating whether an object can be grasped by each pixel photographed by the camera 18." and ¶0052 via "For example, each pixel of the PLM is a map which shows a value near one when the target object can be grasped if the gripper 16 exists at the location (x, y) corresponding to the pixel, and shows a value near 0.5 or a value of 0.5 or less when the target object cannot be grasped" and ¶0083 via "Next, the estimation part 106 may select information with a high score from the data of the depth and posture of the graspable gripper 16 output by the learning model and output the information through the output part 108 (step S22). For example, the output maps of the PLM and the PCM in themselves are referred to as the score." Additionally, see ¶0088 via "a point (x, y) where the output value of the PLM is the highest is extracted as the score…graspable location and posture may be extracted by evaluating the output values of the PLM and the PCM by using predetermined evaluation functions…location and posture having the highest product of the output values of the PLM and the PCM may be simply output, or the location and the posture having the highest value of a weighted average of the PLM and the PCM obtained by a predetermined weighting may be output").
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Kusano's multiple maps and appropriateness scores in order to improve the selection of graspable locations/postures and make it possible to utilize a higher-dimensional DOF which would allow for more mobility: "it is possible to estimate information representing a graspable state for higher-dimensional degree-of-freedom by using the supervised data based on other parameters such as a bending state of the claw of the gripper 16 at a joint, and a distance between the claws." [Kusano ¶0089].
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Nishina et. al. (US 20190143507 A1 -- Corresponding to US10888995B2 in IDS) in view of Hoffman et. al. (US 20220234208 A1).
Regarding Claim 7, Nishina discloses the holding parameter estimation device according to Claim 6.
However, Nishina does not explicitly disclose, but Hoffman—who is directed towards image based guidance for picking—discloses: wherein, the controller is configured to assume an intersecting point between a virtual straight line and a perpendicular line as an adjusted first center position in response to a calculated first center position being deviated from the virtual straight line or intersecting points between virtual straight lines and perpendicular lines as the center positions in response to the first center positions being deviated from the virtual straight lines, the virtual straight line passing through a center position of the end effector area in an opening-closing direction of the at least one holder, the virtual straight lines passing through center positions of the end effector area in opening-closing directions of the at least one holder, the perpendicular line being drawn from the first center position to the virtual straight line, the perpendicular lines being drawn from the first center positions to the respective virtual straight lines, and estimate the position of the at least one holder corresponding to the opening width (See at least ¶0019 via "… (c) calculating a deviation of an orientation of the wire gripper from being parallel to the second portion of the wire based on the camera images taken in step (a); (d) controlling the robot to rotate the wire gripper so that the orientation of the wire gripper matches the orientation of the second portion of the wire…" and "…(k) determining that the distance calculated in step (j) is less than the first threshold; and (l) controlling a pair of gripper fingers of the wire gripper to move to respective positions where the gripper fingers constrain displacement of the second portion of the wire in directions perpendicular to an axis of the wire in response to step (k)…". Additionally see ¶0059. *Wherein, Hoffman is using image guidance to determine and correct a deviation of the gripper/holder. Additionally, Hoffman is determining a center position of a gripper and the relative position with respect to a reference line based on images from a camera, from which the distance between the center of the tip of the gripper and the target is calculated.---Furthermore, see 112(b) rejections).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Hoffman in order to correct for deviations detected by the cameras, as "Capturing images having multiple viewpoints, such as using two or more cameras, enables accurate positioning of the tip 9 of wire gripper 30 relative to wire 11 and then relative to wire contact 3." [Hoffman ¶0055] and so the target item (in this case, a wire) can properly be gripped: "Given the estimated deviation in orientation, robot movement commands are executed to rotate the wire gripper 30 until its orientation matches the orientation of wire 11 (step 108). The orientations match if the axes of the wire gripper 30 and wire 11 are parallel or at a sufficiently small angle that wire 11 may enter the space between gripper fingers 52a and 52b when open wire gripper 30 is lowered." [Hoffman ¶0058].
Regarding Claim 8, Nishina discloses the holding parameter estimation device according to Claim 7.
However, Nishina does not explicitly disclose, but Hoffman discloses: wherein, the controller is configured to search for the approach area in a part of the possible existence area in response to the approach area not existing upon displacement of the at least one holder from the first center position toward the center position of the end effector area or approach areas in a part of the possible existence areas in response to the approach areas not existing upon displacement of the respective holders from the first center position toward the center position of the end effector area, the part of the possible existence area being outside the first center position in a direction parallel to the virtual straight line, the part of the possible existence areas being outside the first center positions in a direction parallel to the virtual straight lines (See at least ¶0019 via "… (c) calculating a deviation of an orientation of the wire gripper from being parallel to the second portion of the wire based on the camera images taken in step (a); (d) controlling the robot to rotate the wire gripper so that the orientation of the wire gripper matches the orientation of the second portion of the wire…" and "…(k) determining that the distance calculated in step (j) is less than the first threshold; and (l) controlling a pair of gripper fingers of the wire gripper to move to respective positions where the gripper fingers constrain displacement of the second portion of the wire in directions perpendicular to an axis of the wire in response to step (k)…". *Wherein the deviated approach is unavailable because it would not result in picking the item, and so the searching for the new approach is the approach/realignment and correction to be able to accurately pick the item. *Additionally, when the initial approach is unavailable because of deviation, Hoffman searches for an alternative approach, and also determines the gripper's deviation based on the images from the camera and move/realigns the gripper to a corrected position in order to properly grip the target. --- furthermore, see 112(b) rejections).
Therefore, it would have been obvious to one of ordinary skill in the art prior to the effective filing date of the given invention to modify Nishina in view of Hoffman in order to correct for deviations so the target item (in this case, a wire) can properly be gripped and the process can continue: "Given the estimated deviation in orientation, robot movement commands are executed to rotate the wire gripper 30 until its orientation matches the orientation of wire 11 (step 108). The orientations match if the axes of the wire gripper 30 and wire 11 are parallel or at a sufficiently small angle that wire 11 may enter the space between gripper fingers 52a and 52b when open wire gripper 30 is lowered." [Hoffman ¶0058].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAYLA RENEE DOROS whose telephone number is (703)756-1415. The examiner can normally be reached Generally: M-F (8-5) EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.D./Examiner, Art Unit 3657
/ABBY LIN/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights