DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed on 07/13/2026 has been entered.
Response to Amendment
The amendment filed 07/13/2026 is being entered. Claims 1-28 are canceled. Claims 29, 37, 47, 48, and 49 are amended. Claim 51 is a new claim. Claims 29-51 are pending, and rejected as detailed below.
Response to Arguments
35 U.S.C. § 102 Rejections
Applicant argues that Bukhari fails to disclose a volume of interest (VOI) that is "generated based on a size of a payload to be placed at the target drop zone" as claimed in amended claims 29 and 37. At most, Bukhari discloses detecting a volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest. See Bukhari para. [0044]. While Bukhari detects a volume of available space and compares it to the dimensions of the element, Bukhari does not form the volume based upon the payload size. See Bukhari para. [0044]. Indeed, Bukhari confirms that the dimensions of the element are used only to test an already-detected volume, and not to generate that volume: "the dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume." See Bukhari para. [0062]. This is distinct from the VOI of the claimed invention, as claimed in amended independent claims 29 and 37, which is generated based upon the size of the payload. See Specification paras. [0087], [0090]; Fig. 4. Given that the VOI the invention as claimed in claims 29 and 37 is generated based on the payload size and is distinct from the volume detected from the region of interest in Bukhari, Bukhari fails to show every limitation of claims 29 and 37, as claimed.
Similarly, Bukhari fails to disclose "generating a volume of interest (VOI) at the target drop zone based on a size of a payload to be delivered" as claimed in amended claim 47. As noted above, Bukhari at most detects a volume of available space from the presence and absence of 3D data within a region of interest, and thereafter compares that detected volume to the dimensions of the element. See Bukhari paras. [0044], [0062]. Bukhari does not generate the volume based upon the size of the payload. This is distinct from claim 47 as amended, in which the VOI is generated based on the size of the payload to be delivered. See Specification paras. [0087], [0090]; Fig. 4. Given that Bukhari does not disclose generating a VOI based on the payload size, Bukhari fails to show every limitation of claim 47 as claimed.
The additional passages of Bukhari of record do not remedy these deficiencies. Bukhari para. [0049] describes detecting rack components, such as vertical pillars 94, 95 and horizontal beams 96, 97, and the volumes bounded by those components; those volumes are defined by the physical rack structure, and are not generated based on the size of the payload. Bukhari paras. [0060] and [0062] describe detecting a rack containing a volume, identifying the top pallet of a stack in a partially-filled rack, and placing an element at an appropriate location. These passages, at most, describe recognizing which portions of a rack are occupied and which remain available across successive placements, and say nothing about how a volume of interest is generated. To the extent the rejection relies on inherency, the missing feature must be necessarily present, not merely probable or possible. See MPEP § 2112. Nothing in Bukhari's detection of sensor-observed volumes, or its tracking of occupied rack locations, necessarily involves generating a VOI based on the size of the payload.
Bukhari fails to teach, suggest or disclose certain elements of the invention set forth in amended independent claims 29, 37 and 47, as noted above. Therefore, it is believed that claims 29, 37 and 47 and claims 30-36, 38-46 and 48-51 dependent thereon, respectively, are allowable over the cited reference, and reconsideration of the rejections of claims 1-5 under 35 U.S.C. 102(a)(1) as being anticipated by Bukhari, is respectfully requested.
New claim
Applicant argues that new claim 51 further recites that, to determine if an obstruction is detected within the volume of interest, the drop zone obstruction detection system is configured to determine whether an obstruction would intrude into the VOI if the payload were placed at the target drop zone. Support for new claim 51 is found at least at paragraph [0095] and steps 510-512 of FIG. 5 of the specification as filed. Bukhari fails to teach suggest or disclose the elements of new claim 51.
Response:
Applicant’s arguments, as amended herein, with respect to the rejections of claims 29, 37, and 47 under 35 U.S.C. §102 have been fully considered and persuasive. Even though Bukhari generates VOI based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, Bukhari does not disclose a volume of interest (VOI) is generated based on a size of a payload to be placed at the target drop zone.
Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection for claims 29, 37, and 47 under 35 U.S.C. §103 is made in view of previously applied reference Bukhari, and further in view of Shteinfeld (US 10262222 B2). In particular, the amendments to claims 29, 37, and 47 are addressed in the instant office action. Furthermore, new claim 51 is also addressed in the instant office action in regards to the combination of Bukhari and Shteinfeld.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 51 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 51 merely states the same limitation of claim 29 (line 8) and does not further narrow what is claimed in claim 29. Following italic, bold, and underline features illustrate the similarities between the claim 29 and claim 51.
Claim 29; process at least some of the sensor data at the target drop zone to determine if an obstruction is detected within the volume of interest
Claim 51; wherein to determine if an obstruction is detected within the volume of interest, the drop zone obstruction detection system is configured to determine whether an obstruction would intrude into the volume of interest if the payload were placed at the target drop zone.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 29-51 are rejected under 35 U.S.C. 103 as being unpatentable over Bukhari (US 20210349468 A1), and further in view of Shteinfeld (US 10262222 B2).
Regarding claim 29, Bukhari teaches (Currently Amended) A robotic vehicle (Bukhari, at least one para. 0001; “This specification relates generally to examples of a mobile robot configured to detect elements in an environment.”), comprising:
a chassis and a manipulatable payload engagement portion (Bukhari, at least one para. 0031; “Robot 10 includes a body 12 having wheels (not shown) to enable robot 10 to travel across a surface, such as the floor of a warehouse, a factory, or other terrain. Robot 10 also includes a support area 15 configured to support the weight of an element, such as a pallet, a container, or any other device to be manipulated, using an end-effector 16.”);
sensors configured to acquire real-time sensor data (Bukhari, at least one para. 0038; “Referring also to the block diagram of FIG. 3, one or more sensors 36a, 36b, 36c, 36d, 36e, and 36f are located on robot 10 for use in detecting the location of the robot itself, for detecting an element to pick-up, and/or for detecting a location on which to place—for example, to stack—the element.”); and
a drop zone obstruction detection system comprising computer program code executable by at least one processor to evaluate the sensor data to (Bukhari, at least one para. 0029; “A control system, which may include one or more processing devices examples of which are described herein, is configured—for example programmed—to control the end-effector, the robot body, or both the end-effector and the robot body to move in three or more degrees of freedom—for example, in at least four degrees of freedom—to stack the first element on top of the second element or to lift the first element off of the second element and move it away from the second element.”):
identify a target drop zone (Bukhari, at least one para. 0044; “The operations may include detecting a rack containing a volume in which to place or from which to remove an element, pointing a sensor above a bottom of the rack to a region of interest”);
generate a volume of interest (VOI) at the target drop zone (Bukhari, at least one para. 0044; “detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, and determining, based on the 3D data and dimensions of the element, whether the volume is large enough to fit the element.”), wherein the VOI is generated based on a size of a payload to be placed at the target drop zone; and
process at least some of the sensor data at the target drop zone to determine if an obstruction is detected within the volume of interest (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification.”, wherein components 94-97 are the obstructions) and (Bukhari, at least one para. 0060; “Operations 110 are described with respect to a rack and a pallet element; however, they may be performed for any appropriate type of volume that holds any appropriate type of element. Operations 110 include detecting (111) a rack containing the volume. The rack may be detected as described by executing operations 70 of FIG. 4. For example, a sensor on a robot captures image data representing components of the rack in a region of interest containing the rack; the image data is filtered to produce filtered data having less of an amount of data than the image data; components of the rack—such as horizontal and vertical beams—are identified by analyzing the filtered data using a deterministic process; and the rack is detecting based on the components. For example, the components are compared to predefined rack configurations in the database and the one that matches is identified as the rack. This information may be used to determine where to look for a volume in which to place an element. In this regard, the rack may be partially full and, therefore, there is still a need to identify regions of the rack that are available to receive an element.”, wherein the obstruction can be already placed elements).
Even though Bukhari teaches about “the target drop zone” being generated via detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, Bukhari does not explicitly teach wherein the VOI is generated based on a size of a payload to be placed at the target drop zone;
Shteinfeld, in the same field of endeavor (Shteinfeld, background; “The present invention relates to a method and a system for measuring dimensions of a target object.”) teaches wherein the VOI is generated based on a size of a payload to be placed at the target drop zone (Shteinfeld, col. 8, lines 40-49; “The point cluster 28a of the target object is then projected onto the z=0 plane (i.e. a plane in which all points have a z-value of zero) as shown in FIG. 3E. Using the projected points of the point cluster 28a a rectangle 32 is selected from a number of geometric shapes, wherein the rectangle is calculated such that it encloses a maximum number of points of the projected point cluster 28a, without exceeding the projected point cluster 28a. Based on the size of the rectangle 32 and the height of point cluster 28a over the ground plane 26a the dimensions, i.e. the volume of the target object 16 is calculated.”);
Bukhari and Shteinfeld are both considered to be analogous to the claimed invention because both of them are in the same field of generating a volume of interest as the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to have generation of volume of interest of the Bukhari with teaching of Shteinfeld. One of the ordinary skill in the art would have been motivated to make this modification so that the payload can be placed within the target drop zone without interfering with obstructions which can be structural components within the target drop zones and/or existing payload. Furthermore, all the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.
Regarding claim 30, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the drop zone obstruction detection system is configured to generate control signals to cause the payload engagement portion to drop a pallet in the target drop zone when no obstruction in the target drop zone is determined (Bukhari, at least one para. 0062; “Operations 110 include determining (114), based on the 3D data that was received and the lack thereof in a region and dimensions of the element, whether the volume is large enough to fit the element. In this regard, the dimensions of the element may be detected as described herein or may be programmed into the robot or otherwise known beforehand. The dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume. The element may be aligned (116) within the volume.”).
Regarding claim 31, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the drop zone obstruction detection system is configured to generate control signals to cause the payload engagement portion to hold a pallet when an obstruction in the target drop zone is determined (Bukhari, at least one para. 0062; “If the rack is empty, this may be done based on knowledge of the rack's geometry. If the rack is not empty—for example, the rack may contain a stack of pallets—the top pallet in the stack may be identified using operations 70 or any other appropriate methodology. The robot then moves (117) the element into the rack and places it at an appropriate location, for example, on the rack itself or a stack held in the rack.”).
Regarding claim 32, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the drop zone obstruction detection system is configured to extract and segment at least one feature within the drop zone based on at least some of the sensor data to determine whether an obstruction is within the VOI (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification.”, wherein each of the components 94-97 are the obstructions segments features).
Regarding claim 33, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the robotic vehicle is an autonomous mobile robot forklift, an autonomous mobile robot tugger, or autonomous mobile pallet truck (Bukhari, at least one para. 0027; “Autonomous vehicles used as examples herein include mobile robots (or simply “robots”); however, any appropriate type of autonomous vehicle may be used including, but not limited to, self-driving machinery or stationary robots. The elements used as examples herein include pallets and containers; however, any appropriate types of elements may be used including, but not limited to, boxes, racks, crates, or bins. As noted, an example pallet includes a flat transport structure that supports goods during lifting. A pallet also includes a mechanism called a socket that the robot's end-effectors may enter and that are used to connect to, and to lift, the pallet. ”).
Regarding claim 34, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the sensors include payload area sensors and/or fork tip sensors (Bukhari, at least one para. 0040; “sensors 36a, 36c may be located on, and movable with, the end-effector. A sensor located here enables the robot to detect and to image elements, such as a pallet or container, that are located directly in front of the end-effector. Data from such sensors enables the end-effector to identify sockets in the element and, therefore, facilitates entry of the end-effector into the sockets in the element. In some examples, one or more sensors may be located on the body of the robot. For example, one or more sensors 36d may be located at a mid-point of the robot, one or more sensors 36b may be located at a bottom of the robot, and/or one or more sensors 36e may be located above sensors 36d. One or more sensors 36f may be located at the top of the robot.”).
Regarding claim 35, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the sensor data includes point cloud data (Bukhari, at least one para. 0051; “one or more of sensors 36a, 36b, 36c, 36d, 36e, or 36f may be directed at the element to obtain image data representing components of the element. In some implementations, the image data may be 3D data captured by sensors such as a 3D camera or multiple LIDAR sensors. In an example, the image data defines a point cloud.”).
Regarding claim 36, Bukhari teaches (Previously Presented) The vehicle of claim 29, wherein the drop zone is a floor, a drop table or conveyor, rack shelving, a top of a pallet already dropped, or a bed of an industrial cart (Bukhari, at least one para. 0060; “Operations 110 are described with respect to a rack and a pallet element; however, they may be performed for any appropriate type of volume that holds any appropriate type of element.”).
Regarding claim 37, Bukhari teaches (Currently Amended) A drop zone obstruction detection method for use by a robotic vehicle (Bukhari, at least one para. 0001; “This specification relates generally to examples of a mobile robot configured to detect elements in an environment.”), comprising:
providing a robotic vehicle having a chassis and a manipulatable payload engagement portion (Bukhari, at least one para. 0031; “Robot 10 includes a body 12 having wheels (not shown) to enable robot 10 to travel across a surface, such as the floor of a warehouse, a factory, or other terrain. Robot 10 also includes a support area 15 configured to support the weight of an element, such as a pallet, a container, or any other device to be manipulated, using an end-effector 16.”), sensors configured to acquire real-time sensor data (Bukhari, at least one para. 0038; “Referring also to the block diagram of FIG. 3, one or more sensors 36a, 36b, 36c, 36d, 36e, and 36f are located on robot 10 for use in detecting the location of the robot itself, for detecting an element to pick-up, and/or for detecting a location on which to place—for example, to stack—the element. ”), and a drop zone obstruction detection system comprising computer program code executable by at least one processor (Bukhari, at least one para. 0029; “A control system, which may include one or more processing devices examples of which are described herein, is configured—for example programmed—to control the end-effector, the robot body, or both the end-effector and the robot body to move in three or more degrees of freedom—for example, in at least four degrees of freedom—to stack the first element on top of the second element or to lift the first element off of the second element and move it away from the second element.”); and
the drop zone obstruction system:
identifying the target drop zone (Bukhari, at least one para. 0044; “The operations may include detecting a rack containing a volume in which to place or from which to remove an element, pointing a sensor above a bottom of the rack to a region of interest”);
generating a volume of interest (VOI) at the target drop zone (Bukhari, at least one para. 0044; “detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, and determining, based on the 3D data and dimensions of the element, whether the volume is large enough to fit the element.”), wherein the VOI is generated based on a size of a payload to be placed at the target drop zone; and
processing at least some of the sensor data at the target drop zone to determine if an obstruction is detected within the volume of interest (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification.”, wherein the components 94-97 are the obstructions).
Even though Bukhari teaches about “the target drop zone” being generated via detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, Bukhari does not explicitly teach wherein the VOI is generated based on a size of a payload to be placed at the target drop zone;
Shteinfeld, in the same field of endeavor (Shteinfeld, background; “The present invention relates to a method and a system for measuring dimensions of a target object.”) teaches wherein the VOI is generated based on a size of a payload to be placed at the target drop zone (Shteinfeld, col. 8, lines 40-49; “The point cluster 28a of the target object is then projected onto the z=0 plane (i.e. a plane in which all points have a z-value of zero) as shown in FIG. 3E. Using the projected points of the point cluster 28a a rectangle 32 is selected from a number of geometric shapes, wherein the rectangle is calculated such that it encloses a maximum number of points of the projected point cluster 28a, without exceeding the projected point cluster 28a. Based on the size of the rectangle 32 and the height of point cluster 28a over the ground plane 26a the dimensions, i.e. the volume of the target object 16 is calculated.”);
Bukhari and Shteinfeld are both considered to be analogous to the claimed invention because both of them are in the same field of generating a volume of interest as the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to have generation of volume of interest of the Bukhari with teaching of Shteinfeld. One of the ordinary skill in the art would have been motivated to make this modification so that the payload can be placed within the target drop zone without interfering with obstructions which can be structural components within the target drop zones and/or existing payload. Furthermore, all the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.
Regarding claim 38, Bukhari teaches (Previously Presented) The method of claim 37, further comprising the drop zone obstruction detection system generating control signals to cause the payload engagement portion to drop the pallet in the target drop zone in response to determining that there is no obstruction in the target drop zone (Bukhari, at least one para. 0062; “Operations 110 include determining (114), based on the 3D data that was received and the lack thereof in a region and dimensions of the element, whether the volume is large enough to fit the element. In this regard, the dimensions of the element may be detected as described herein or may be programmed into the robot or otherwise known beforehand. The dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume. The element may be aligned (116) within the volume.”).
Regarding claim 39, Bukhari teaches (Previously Presented) The method of claim 37, further comprising the drop zone obstruction detection system generating control signals to cause the payload engagement portion to hold a pallet in response to determining that there is at least one obstruction in the target drop zone (Bukhari, at least one para. 0062; “If the rack is empty, this may be done based on knowledge of the rack's geometry. If the rack is not empty—for example, the rack may contain a stack of pallets—the top pallet in the stack may be identified using operations 70 or any other appropriate methodology. The robot then moves (117) the element into the rack and places it at an appropriate location, for example, on the rack itself or a stack held in the rack.”).
Regarding claim 40, Bukhari teaches (Previously Presented) The method of claim 37, further comprising the drop zone obstruction detection system extracting and segmenting features within the drop zone based on at least some of the sensor data and determining whether an obstruction is within the VOI (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification.”, wherein the components 94-97 are the obstructions segments features).
Regarding claim 41, Bukhari teaches (Previously Presented) The method of claim 37, wherein the robotic vehicle is an autonomous mobile robot forklift, an autonomous mobile robot tugger, or an autonomous mobile robot pallet truck (Bukhari, at least one para. 0027; “Autonomous vehicles used as examples herein include mobile robots (or simply “robots”); however, any appropriate type of autonomous vehicle may be used including, but not limited to, self-driving machinery or stationary robots. The elements used as examples herein include pallets and containers; however, any appropriate types of elements may be used including, but not limited to, boxes, racks, crates, or bins. As noted, an example pallet includes a flat transport structure that supports goods during lifting. A pallet also includes a mechanism called a socket that the robot's end-effectors may enter and that are used to connect to, and to lift, the pallet. ”).
Regarding claim 42, Bukhari teaches (Previously Presented) The method of claim 37, wherein the one or more sensors comprises at least one LiDAR scanner (Bukhari, at least one para. 0038; “robot 10 may include one or more 3D cameras, one or more light detection and ranging (LIDAR) scanners, one or more optical sensors, one or more sonar sensors, one or more time-of-flight (TOF) sensors, one or more radar sensors, one or more 2D camera sensors, one or more ultrasonic sensors, or any appropriate multiple numbers and/or combination thereof.”).
Regarding claim 43, Bukhari teaches (Previously Presented) The method of claim 37, wherein the one or more sensors comprises at least one stereo camera (Bukhari, at least one para. 0038; “robot 10 may include one or more 3D cameras, one or more light detection and ranging (LIDAR) scanners, one or more optical sensors, one or more sonar sensors, one or more time-of-flight (TOF) sensors, one or more radar sensors, one or more 2D camera sensors, one or more ultrasonic sensors, or any appropriate multiple numbers and/or combination thereof.”).
Regarding claim 44, Bukhari teaches (Previously Presented) The method of claim 37, wherein the sensors include payload area sensors and/or fork tip sensors (Bukhari, at least one para. 0040; “sensors 36a, 36c may be located on, and movable with, the end-effector. A sensor located here enables the robot to detect and to image elements, such as a pallet or container, that are located directly in front of the end-effector. Data from such sensors enables the end-effector to identify sockets in the element and, therefore, facilitates entry of the end-effector into the sockets in the element. In some examples, one or more sensors may be located on the body of the robot. For example, one or more sensors 36d may be located at a mid-point of the robot, one or more sensors 36b may be located at a bottom of the robot, and/or one or more sensors 36e may be located above sensors 36d. One or more sensors 36f may be located at the top of the robot.”).
Regarding claim 45, Bukhari teaches (Previously Presented) The method of claim 37, wherein the sensor data includes point cloud data (Bukhari, at least one para. 0051; “one or more of sensors 36a, 36b, 36c, 36d, 36e, or 36f may be directed at the element to obtain image data representing components of the element. In some implementations, the image data may be 3D data captured by sensors such as a 3D camera or multiple LIDAR sensors. In an example, the image data defines a point cloud.”).
Regarding claim 46, Bukhari teaches (Previously Presented) The method of claim 37, wherein the drop zone is a floor, a drop table or conveyor, rack shelving, a top of a pallet already dropped, or a bed of an industrial cart (Bukhari, at least one para. 0060; “Operations 110 are described with respect to a rack and a pallet element; however, they may be performed for any appropriate type of volume that holds any appropriate type of element.”).
Regarding claim 47, Bukhari teaches (Currently Amended) A drop zone obstruction detection method (Bukhari, at least one para. 0001; “This specification relates generally to examples of a mobile robot configured to detect elements in an environment.”) and (Bukhari, at least one para. 0029; “A control system, which may include one or more processing devices examples of which are described herein, is configured—for example programmed—to control the end-effector, the robot body, or both the end-effector and the robot body to move in three or more degrees of freedom—for example, in at least four degrees of freedom—to stack the first element on top of the second element or to lift the first element off of the second element and move it away from the second element.”), comprising:
providing mobile robot (Bukhari, at least one para. 0031; “Robot 10 includes a body 12 having wheels (not shown) to enable robot 10 to travel across a surface, such as the floor of a warehouse, a factory, or other terrain. Robot 10 also includes a support area 15 configured to support the weight of an element, such as a pallet, a container, or any other device to be manipulated, using an end-effector 16.”) with one or more sensors (Bukhari, at least one para. 0038; “Referring also to the block diagram of FIG. 3, one or more sensors 36a. 36b, 36c, 36d, 36e, and 36f are located on robot 10 for use in detecting the location of the robot itself, for detecting an element to pick-up, and/or for detecting a location on which to place—for example, to stack—the element.”);
identifying a target drop zone (Bukhari, at least one para. 0044; “The operations may include detecting a rack containing a volume in which to place or from which to remove an element, pointing a sensor above a bottom of the rack to a region of interest”);
using the one or more sensors (Bukhari, at least one para. 0038; “sensors 36a. 36b, 36c, 36d, 36e, and 36f ”), collecting point cloud data from locations at or near a drop zone (Bukhari, at least one para. 0051; “one or more of sensors 36a, 36b, 36c, 36d, 36e, or 36f may be directed at the element to obtain image data representing components of the element. In some implementations, the image data may be 3D data captured by sensors such as a 3D camera or multiple LIDAR sensors. In an example, the image data defines a point cloud.”);
generating a volume of interest (VOI) at the target drop zone (Bukhari, at least one para. 0044; “detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, and determining, based on the 3D data and dimensions of the element, whether the volume is large enough to fit the element.”) based on a size of a payload to be delivered; and
performing an obstruction detection analysis based on the point cloud data to determine if there are obstructions in the drop zone (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification.”, wherein the components 94-97 are the obstructions).
Even though Bukhari teaches about “the target drop zone” being generated via detecting the volume based on 3D data within a first part of the region of interest and a lack of 3D data within a second part of the region of interest, Bukhari does not explicitly teach a volume of interest (VOI) at the target drop zone based on a size of a payload to be delivered;
Shteinfeld, in the same field of endeavor (Shteinfeld, background; “The present invention relates to a method and a system for measuring dimensions of a target object.”) teaches a volume of interest (VOI) at the target drop zone based on a size of a payload to be delivered (Shteinfeld, col. 8, lines 40-49; “The point cluster 28a of the target object is then projected onto the z=0 plane (i.e. a plane in which all points have a z-value of zero) as shown in FIG. 3E. Using the projected points of the point cluster 28a a rectangle 32 is selected from a number of geometric shapes, wherein the rectangle is calculated such that it encloses a maximum number of points of the projected point cluster 28a, without exceeding the projected point cluster 28a. Based on the size of the rectangle 32 and the height of point cluster 28a over the ground plane 26a the dimensions, i.e. the volume of the target object 16 is calculated.”);
Bukhari and Shteinfeld are both considered to be analogous to the claimed invention because both of them are in the same field of generating a volume of interest as the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to have generation of volume of interest of the Bukhari with teaching of Shteinfeld. One of the ordinary skill in the art would have been motivated to make this modification so that the payload can be placed within the target drop zone without interfering with obstructions which can be structural components within the target drop zones and/or existing payload. Furthermore, all the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods with no change in their respective functions, and the combination would have yielded predictable results to one of ordinary skill in the art at the time of the invention.
Regarding claim 48, Bukhari teaches (Previously Presented) The method of claim 47, further comprising generating a signal corresponding to a presence or absence of at least one obstruction in the target drop zone (Bukhari, at least one para. 0062; “Operations 110 include determining (114), based on the 3D data that was received and the lack thereof in a region and dimensions of the element, whether the volume is large enough to fit the element. In this regard, the dimensions of the element may be detected as described herein or may be programmed into the robot or otherwise known beforehand. The dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume. The element may be aligned (116) within the volume. If the rack is empty, this may be done based on knowledge of the rack's geometry. If the rack is not empty—for example, the rack may contain a stack of pallets—the top pallet in the stack may be identified using operations 70 or any other appropriate methodology. The robot then moves (117) the element into the rack and places it at an appropriate location, for example, on the rack itself or a stack held in the rack.”).
Regarding claim 49, Bukhari teaches (Currently Amended) The method of claim 47, wherein performing the obstruction detection analysis further comprises:
determining boundaries of the target drop zone by extracting features from the point cloud data (Bukhari, at least one para. 0049; “a rack, such as rack 81 of FIG. 6, may include vertical pillars such as 94, 95 and horizontal beams such as 96, 97. The techniques described herein may be used to detect those components, volumes bounded by those components, or both the components and volumes. The information—for example, data—representing the components, volumes, or both are used for detection, including identification. (wherein 94-97 are the obstructions segments features)”);
comparing the VOI to the boundaries of the drop zone to determine the presence or absence of potential obstructions in the target drop zone (Bukhari, at least one para. 0062; “Operations 110 include determining (114), based on the 3D data that was received and the lack thereof in a region and dimensions of the element, whether the volume is large enough to fit the element. In this regard, the dimensions of the element may be detected as described herein or may be programmed into the robot or otherwise known beforehand. The dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume. The element may be aligned (116) within the volume. If the rack is empty, this may be done based on knowledge of the rack's geometry. If the rack is not empty—for example, the rack may contain a stack of pallets—the top pallet in the stack may be identified using operations 70 or any other appropriate methodology. The robot then moves (117) the element into the rack and places it at an appropriate location, for example, on the rack itself or a stack held in the rack.”).
Regarding claim 50, Bukhari teaches (Previously Presented) The method of claim 47, further comprising determining if an object to be delivered fits within the drop zone based on a comparison of dimensions of the object and the obstruction detection analysis (Bukhari, at least one para. 0062; “Operations 110 include determining (114), based on the 3D data that was received and the lack thereof in a region and dimensions of the element, whether the volume is large enough to fit the element. In this regard, the dimensions of the element may be detected as described herein or may be programmed into the robot or otherwise known beforehand. The dimensions of the element are compared to the size of the volume. If the volume is larger than the element, then it is determined that the element will fit within the volume.”).
Regarding claim 51, Bukhari teaches (New) The vehicle of claim 29, wherein to determine if an obstruction is detected within the volume of interest (Bukhari, at least one para. 0060; “Operations 110 are described with respect to a rack and a pallet element; however, they may be performed for any appropriate type of volume that holds any appropriate type of element. Operations 110 include detecting (111) a rack containing the volume. The rack may be detected as described by executing operations 70 of FIG. 4. For example, a sensor on a robot captures image data representing components of the rack in a region of interest containing the rack; the image data is filtered to produce filtered data having less of an amount of data than the image data; components of the rack—such as horizontal and vertical beams—are identified by analyzing the filtered data using a deterministic process; and the rack is detecting based on the components. For example, the components are compared to predefined rack configurations in the database and the one that matches is identified as the rack. This information may be used to determine where to look for a volume in which to place an element. In this regard, the rack may be partially full and, therefore, there is still a need to identify regions of the rack that are available to receive an element.”, wherein the obstruction can be already placed elements), the drop zone obstruction detection system is configured to determine whether an obstruction would intrude into the volume of interest if the payload were placed at the target drop zone (if an obstruction is detected within the volume of interest, it would obvious that the obstruction would intrude into the volume of interest due to the already placed payload).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UPUL P CHANDRASIRI whose telephone number is (703)756-5823. The examiner can normally be reached M-F 8.30 am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at 571-272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/U.P.C./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
