Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is in response to the amendments filed on May 22, 2026. Claims 1, 6, 7, 12, 13, and 18 are amended. Claims 19 and 20 are added. Claims 1-20 are pending and addressed below.
Response to Arguments
Applicant’s arguments regarding the prior art rejection of claim 1 has been fully considered but are not persuasive.
Applicant argues that Zhang’s picking and placing is predetermined in descending order of volume and independent of binary mask vectors and height maps. However, examiner disagrees as [n0043] of Zhang discloses that the point cloud of each instance, where each object instance is generated by point cloud segmentation, is projected into a binary mask and fed into the Dex-Net network in descending order of volume. Furthermore, [n0042] also states “after the depth camera captures the stacking state of objects …” and grasps the object O. Zhang requires determining a sorted order by traversing all boxes.
Applicant argues that Zhang does not disclose a mask matrix generated based on hypothetical situations. However, Zhang discloses that the four heuristic methods (disclosed in at least [n0059-n0063]) is used within a mask E indicating the preferred placement position (see at least [n0064]). As these heuristics are for figuring a placement of the object, Zhang does disclose a mask matrix generated based on hypothetical situations is obtained by traversing the elements in the height map matrix.
Applicant argues that Zhang’s action set is a one-dimensional vector mask rather than a two-dimensional mask matrix. However, examiner notes that the heuristic algorithms of mask E pertain to a placement of the box that are calculated by at least a feasible position (p,q) (see at least [n0063]), where Zhang further discloses that “E is an L x W binary matrix”, where broadly this is interpreted to be a matrix of at least length and width of the placing area.
Applicant argues that Zhang’s box placement does not pertain to information of other boxes or global comparisons and that there is no logic for global optimal box selection. As the language is newly amended. See the relevant section below regarding this limitation.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-18 are rejected under 35 U.S.C. 103 as being unpatentable over CN113592855A (Zhang) in view of US9333649B1 (Bradski).
Regarding claims 1, 7, and 13, Zhang discloses a palletizing method for unknown incoming materials, a control device, and a non-transitory computer-readable storage medium (NTCRSM) storing instructions, comprising:
detecting, by one or more depth cameras, a size for each of a plurality of boxes in an incoming material area and a stacking pattern on a pallet;
Zhang discloses detecting, by one or more depth cameras ([n0042], “the depth camera acquires the stacking status of objects in the grabbing box B<sub>source</sub>, …”), a size of plurality of boxes in an incoming material area ([n0042], “At the start of the task, the grabbing box B<sub>source</sub> in the grabbing area W<sub>pick</sub> is filled with cube objects of different sizes …”) and a stacking pattern on a pallet ([n0042], “After detecting and calculating the size of the object O to be packed and its placement position in the placement box 5, the second gripping module M<sub>place</sub> grabs the object and puts it into the corresponding position in the placement box B<sub>target</sub>”, see at least Fig. 5b for exemplary stacking pattern).
generating a height map matrix based on the stacking pattern, wherein a value at each element of a plurality of elements in the height map matrix indicates a height of the stacking pattern at a position on the pallet corresponding to the element;
See Fig. 4 of Zhang, where the figure is disclosed to take the state of the object and height map of the target workspace (comprising the stacking pattern). Zhang further discloses that a three-dimensional space is discretized into a two-dimensional grid with height being expressed as the positive value of the length-width coordinate of each placement action ([n0050]).
traversing the boxes in the incoming material area, and for each of the boxes:
traversing the elements in the height map matrix to obtain a mask matrix generated based on hypothetical situations that a target vertex of the box rests on a position corresponding to each of the elements, wherein array dimensions of the mask matrix and the height map matrix are the same; and
Zhang discloses traversing elements in the height map matrix ([n0050], “For an L X W X H box, its space is discretized into an L×W grid Z<sup>+</sup>∈[0,H], which is used as a height map describing the current state of the box.”) to obtain a mask matrix (see at least Fig. 6b and [n0064]), where [n0064] describes forming a mask E from feasible positions (p,q) providing a height h(p,q)) generated based on hypothetical situations that a target vertex of the box rests on a position corresponding to each of the elements (see Fig. 6(a) of Zhang and the “extreme point algorithm” in [n0062]), wherein array dimensions of the mask matrix and the height map matrix.
determining reward function values corresponding to one or more of elements in the mask matrix with an element value of 1, wherein the element value being 1 indicates that first constraint conditions are satisfied when the target vertex of the box rests on a position corresponding to one of the elements in the height map matrix;
Zhang discloses combining the heuristic methods (such as the extreme point heuristic disclosed above) to form a mask matrix indicating priority placements ([n0064]), where determined reward function values corresponding to one or more elements in the mask matrix ([n0064], “reward H”), where the mask matrix disclosed is a binary matrix where the heuristics are incorporated into the deep reinforcement learning.
obtaining a largest reward function value of reward function values corresponding to all elements with an element value of 1 in all mask matrices of the boxes;
While Zhang does not explicitly disclose obtaining a largest reward function value of reward function values corresponding to all elements with an element value of 1 in all mask matrices of the boxes, one of ordinary skill in the art would find it obvious that the largest reward function value is obtained as Zhang discloses the consideration of all box volumes (see at least [n0081]) when using a strategy of maximum space utilization, where the strategy produces efficient outputs (see at least [n0085-n0086] and Figs. 8a-8c, 9a-9c, and 10a-10c, where Fig. 10a achieves a higher space utilization).
determining a box number and a box placement posture corresponding to the largest reward function value of the reward function values; and
Zhang discloses box numbers for observing inefficiencies (see Fig. 5(b) and 5(c)) and box placement postures corresponding to a largest reward function value of the reward function values ([n0045]).
controlling a robotic arm to perform palletizing of a target box corresponding to the box number among the boxes, according to the box number and the box placement posture.
While Zhang discloses controlling a robot arm to perform palletizing according to the box placement posture (see at least [n0019], [n0074]), Zhang does not disclose controlling a robotic arm according to the box number. However, from a similar field of endeavor, Bradski discloses a robotic manipulator that identifies grasp points, motion paths, and drop-off locations of boxes. Specifically, Bradski discloses the use of a variable size queue of boxes (column 12, lines 55-65) that identifies a set of boxes to act as candidates for next box to be stacked.
One of ordinary skill in the art would find it obvious, prior to the applicant’s effective filing date, to combine the system of Bradski to Zhang as the queue disclosed by Bradski would allow for determining a best box placement in response to the current state of the environment, further improving the operation of the robotic arm.
Regarding, claims 2, 8, and 14, with all of the limitations of claims 1, 7, and 13, the method, device, and NTCRSM further comprises:
wherein traversing the elements in the height map matrix to obtain a mask matrix generated based on hypothetical situations that a target vertex of the box rests on a position corresponding to each of the elements comprises:
Zhang discloses combining the heuristic methods (such as the extreme point heuristic disclosed above) to form a mask matrix indicating priority placements ([n0064]), where determined reward function values corresponding to one or more elements in the mask matrix ([n0064], “reward H”), where the mask matrix disclosed is a binary matrix where the heuristics are incorporated into the deep reinforcement learning.
in one of the hypothetical situations when the target vertex of the box rests on a position corresponding to any element of the height map matrix, determining whether the box satisfies the first constraint conditions;
Zhang discloses the hypothetical situation when the target vertex of the box (extreme point) rests on a position corresponding to any element of the heigh map matrix, where a determination is made that the constraint is satisfied (see “determining reward function values ... of claim 1).
in response to the box satisfying the first constraint conditions, setting an element value of the element to 1;
in response to the box not satisfying the first constraint conditions, setting the element value of the element to 0; and
Zhang discloses that a mask matrix is formed to indicate the priority placements based on the heuristic methods disclosed, where the extreme point heuristic is one of the disclosed methods.
traversing the elements in the height map matrix to obtain the mask matrix.
See the limitations of claims 1, 7, and 13 regarding “traversing the elements …” and “determining reward function values…”.
Regarding, claims 3, 9, and 15, with all of the limitations of claims 2, 8, and 14, the method, device, and NTCRSM further comprises:
wherein the first constraint conditions comprise:
a stacking pattern stability after placement of the box exceeding a stability threshold,
Zhang discloses a stacking pattern stability after placement combined with the reinforcement algorithm, where said stacking pattern stability after placement restricts actions to those to be within a stability threshold ([n0055]).
the box being reachable by the robotic arm, and
While Zhang discloses a robotic arm able to reach boxes (see Fig. 3 of Zhang), Zhang does not disclose the box being reachable by the robotic arm as a constraint condition. However, one of ordinary skill in the art would find it obvious to constrain the robot arm’s operations to be boxes reachable by the robotic arm as unreachable boxes cannot be acted on by the arm.
no colliding between the robotic arm and boxes on the pallet when the robotic arm is placing the box on the pallet.
While Zhang does not explicitly disclose a constraint condition where collisions between the robotic arm and boxes on the pallet are to not occur during placing the box on the pallet,
Bradski discloses a collision-free path planning through using constraints and cost functions (Bradski, column 22, line 62 to column 23, line 12).
One of ordinary skill in the art would find it obvious, prior to the applicant’s effective filing date, to combine the collision avoidance of Bradski to the system of Zhang as collision to the boxes on the pallet may result in undesirable configuration, where boxes may fall off of the pallet and proceed to both no longer be represented by stored data structures such as the height map and affect operational efficiency due to the collision.
Regarding, claims 4, 10, and 16, with all of the limitations of claims 1, 7, and 13, the method, device, and NTCRSM further comprises:
wherein determining the reward function values corresponding to one or more of elements in the mask matrix with the element value of 1 comprises:
traversing the one or more elements in the mask matrix whose element value is 1;
Zhang discloses that the mask matrix is a binary mask ([n0064]) with a reward H assigned for elements of the mask matrix.
performing analysis using a cost function of a heuristic strategy and second constraint conditions to obtain the reward function values corresponding to the one or more elements with a value of 1.
While Zhang discloses performing analysis using a cost function of a heuristic strategy (see Table 1 of Zhang and Figs. 8(a) to Fig 10(c), where the figures correspond) to obtain reward function values corresponding to the one or more elements with a value of 1, where the reward function values obtained are from the highest rewarding algorithm ([n0082]), Zhang does not explicitly disclose the use of second constraint conditions to obtain the reward function values. However, one of ordinary skill would find it obvious to try a second constraint condition in performing analysis by cost function as Zhang already discloses the use of a constraint condition (see at least constraints of claims 3, 9, and 15) to better determine an optimal reward algorithm for palletization, where the second constraint may provide more desirable output depending on the choice of constraint.
Regarding, claims 5, 11, and 17, with all of the limitations of claims 3, 9, and 15, the method, device, and NTCRSM further comprises:
wherein the second constraint conditions comprise one or more of conditions as follows:
whether the box will be placed close to other boxes on the pallet; and
While Zhang does not explicitly disclose second constraint conditions such as the ones above, from a similar field of endeavor, Bradski discloses a plan for a robotic device in order to achieve certain configurations within a target location for multiple objectives such as “dense packing with minimal air gaps in between boxes” (box will be placed close to other boxes on the pallet).
One of ordinary skill in the art would find it obvious to combine the system of Bradski to the system of Zhang as a densely packed heuristic would provide another heuristic to modify the deep reinforcement learning system of Zhang, advantageously allowing for more strategies to employ and learn from.
Regarding, claims 6, 12, and 18, with all of the limitations of claims 1, 7, and 13, the method, device, and NTCRSM further comprises:
wherein controlling the robotic arm to perform the palletizing of the target box corresponding to the box number among the boxes, according to the box number and the box placement posture comprises:
controlling an end effector of the robotic arm to move the target box corresponding to the box number; and
In light of the rationale of the limitation “controlling a robotic arm …” of claim 1 above, the queue of Bradski is a queue of boxes to be considered prior to movement, where the selected box is moved (see Figs. 6A, 6C, and 6D of Bradski).
controlling the end effector to place the target box on the pallet according to the placement posture.
See at least one of Figs. 8(a), 8(b), 8(c), 9(a), 9(b), 9(c), 10(a), 10(b), and 10(c) of Zhang, where the figures are all exemplary placement configurations based off of the placement posture of the heuristic functions.
Regarding claim 19, with all of the limitations of claim 1, the method further comprises:
wherein the height map matrix is a two-dimensional array of distance values measured from the pallet to a top of uppermost boxes on the pallet;
[n0050] of Zhang, “For an L X W X H box, its space is discretized into an L×W grid Z<sup>+</sup>∈[0,H], which is used as a height map describing the current state of the box.”
the elements in the mask matrix have values of either 1 or 0, and the elements in the mask matrix are derived from the elements in the height map matrix; and
See at least [n0064] of Zhang, where the mask matrix of the dimensions of the area L x W is disclosed to be a binary matrix.
the array dimension refer to length, width, and number of the elements in the mask matrix.
See the rationale of claim 1 regarding the dimensions of the mask matrix and the height map matrix being the same.
Regarding claim 20, with all of the limitations of claim 1, the method further comprises:
wherein the height of the stacking pattern is voxelized,
See at least Fig. 6(b), where it appears that the grid is voxelized.
one box corresponds to at least one reward function value, and
See at least the rationale of claim 1 above regarding determining reward function values.
the placement posture refers to placement position and orientation of the box.
See at least Fig. 8(a), where the boxes placed are of specific position and orientation.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAEWOOK JUNG whose telephone number is (571)272-5470. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached on (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.J./Examiner, Art Unit 3656
/WADE MILES/Supervisory Patent Examiner, Art Unit 3656
