Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 4-5, 8-10, 14, 17, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US20220340171A1 (Halder).
Regarding claim 1, Halder discloses a perception system of a material handling vehicle comprising:
at least one processor;
[0043], Autonomous vehicle management system 105 may be implemented using software only, hardware only, or combinations thereof. The software may be stored on a non-transitory computer readable medium (e.g., on a memory device) and may be executed by one or more processors (e.g., by computer systems) to perform its functions.
one or more front facing sensors and one or more rear facing sensors designed to capture sensor data of an environment surrounding the material handling vehicle, wherein the at least one processor is designed to identify at least one feature from the sensor data; and
Halder discloses one or more front facing sensors ([0041], “For example, cameras are capable of generating highly detailed images of the environment the path/road or objects are within it,”) and one or more rear facing sensors ([0065], “For example, planning subsystem 220 can command a LIDAR sensor to narrow its range of sensing from a three-hundred and sixty-degree (360°) view to a narrower range that includes a specific object to be sensed and/or tracked in greater detail by the LIDAR system. In this way, the consistent internal map is updated based on feedback from and under the control of planning subsystem 220.”) designed to capture sensor data of an environment surrounding the material handling vehicle, wherein the at least one processor is designed to identify at least one feature from the sensor data (see at least [0057]).
a feedback device of the material handling vehicle, wherein the feedback device is designed to perform a feedback operation based at least in part on one or more signals from the perception system.
See Fig. 2A, where the perception system 215 of the figure is within a feedback operation of the autonomous vehicle.
Regarding claim 2, with all of the limitations of claim 1, the system further comprises:
wherein to identify the at least one feature from the sensor data the at least one processor is further designed to:
apply a computer vision model to the sensor data to identify the at least one feature, wherein the sensor data includes still or continuous images; and
[0059], “For example, perception subsystem 215 may use a convolutional neural network (CNN) to perform object detection and object classification based upon the sensor data. During a training phase, the CNN may be trained using labeled training data comprising sample images of a vehicle's environment and corresponding ground truth classifications. Labeled data generally includes a group of samples that have been tagged with one or more labels, where the labels represent known results (e.g., ground truth classification, etc.) for the training input samples. Labeling can also be used to take a set of unlabeled data and augment each piece of that unlabeled data with meaningful tags that are informative. A CNN model or other AI/machine learning model built based upon training may then be used in real time to identify and classify objects in the environment of autonomous vehicle 100 based upon new sensor data received from sensors 110.”
determine that the at least one feature matches a type of target.
[0059], “A CNN model or other AI/machine learning model built based upon training may then be used in real time to identify and classify objects in the environment of autonomous vehicle 100 based upon new sensor data received from sensors 110.”
Regarding claim 4, with all of the limitations of claim 2, the system further comprises:
wherein to apply the computer vision model to the sensor data the at least one processor is further designed to apply one or more artificial intelligence algorithms to the sensor data to identify at least one feature from the sensor data.
[0059], “A CNN model or other AI/machine learning model built based upon training may then be used in real time to identify and classify objects in the environment of autonomous vehicle 100 based upon new sensor data received from sensors 110.”
Regarding claim 5, with all of the limitations of claim 1, the system further comprises:
determine a distance between the material handling vehicle and the at least one feature; and
generate the one or more signals when the distance between the material handling vehicle and at least one feature is less than or equal to a threshold distance value.
[0061], “Examples of these considerations include, without limitation: always stay within the lane, maintain certain distance from any object at all time, a dump truck is not to make more than a 30 degree turn, a loader B is not to climb over a predetermined grade more than 15 degrees, etc.”, where maintaining a certain distance from any object at all time requires both determining a distance between the vehicle and the feature and generating one or more signals to maintain the distance based on a threshold distance value.
Regarding claim 8, with all of the limitations of claim 1, the system further comprises:
wherein the feedback operation includes outputting an auditory alert, a visual alert, a tactile alert, or combinations thereof.
[0060], “As another example, the plan generated by planning subsystem 220 may include planned actions with respect to accessories of autonomous vehicle 100, such as turning indicators or lights on or off, producing one or more sounds (e.g., alarms), and the like.”
Regarding claim 9, Halder discloses a perception system of a material handling vehicle comprising:
a perception system, comprising:
a plurality of sensors, wherein each sensor of the plurality of sensors is designed to capture sensor data associated with an environment of the material handling vehicle; and
See Fig. 3 of Halder, where an example set of sensors is provided ([0039]).
at least one processor in communication with the plurality of sensors, wherein the at least one processor is designed to identify at least one target in the environment of the material handling vehicle based at least in part on the sensor data; and
See the citation of claim 1.
a vehicle controller in communication with the perception system, wherein the vehicle controller is designed to initiate a response based at least in part on the at least one target identified in the environment surrounding the material handling vehicle.
See the citation of claim 1.
Regarding claim 10, with all of the limitations of claim 9, the vehicle further comprises:
wherein the at least one processor is further designed to:
output at least one signal to the vehicle controller to initiate the response based at least in part on the at least one target and further based at least in part on a speed of the material handling vehicle, a heading of the material handling vehicle, a distance of the material handling vehicle to the at least one target, or combinations thereof.
See the citation of claim 5.
Regarding claim 14, XXX discloses a method of operating a material handling vehicle comprising:
capturing sensor data via at least one sensor of the material handling vehicle;
See the citation of claim 1 regarding “one or more front facing sensors …”.
identifying a target from the captured sensor data; and
See the citation of claim 1 regarding “one or more front facing sensors …”.
performing at least one feedback operation by at least one feedback device of the material handling vehicle based at least in part on a type of the target and at least one of a distance metric or a speed metric of the material handling vehicle.
See the citation of claim 1 regarding “a feedback device of the material handling vehicle…”.
Regarding claim 15, with all of the limitations of claim 14, the method further comprises:
determining a distance between the material handling vehicle and the target, wherein the distance metric is associated with the distance between the material handling vehicle and the target being less than or equal to a threshold distance value.
See the citation of claim 5.
Regarding claim 17, with all of the limitations of claim 14, the method further comprises:
wherein performing the at least one feedback operation at the at least one feedback device further comprises: outputting an audible alert, a visual alert, tactile feedback, or combinations thereof.
See the citation of claim 8.
Regarding claim 20, with all of the limitations of claim 14, the method further comprises:
wherein identifying the target from the captured sensor data further comprises:
analyzing the captured sensor data via at least one computer vision model to distinguish the target from background images, wherein the captured sensor data includes still or continuous images of the captured sensor data; and
comparing an image of the target to a set of images of a plurality of targets to determine the type of target.
See the citation of claim 2.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 6-7, 11-13, 16, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over US20220340171A1 (Halder) in further view of US20160350601A1 (Grauer).
Regarding claim 3, with all of the limitations of claim 2, the system further comprises:
wherein the type of target includes a stop sign, a cross aisle, an end of an aisle, or an aisle sign.
While Halder does not explicitly disclose that the type of target includes a stop sign, a cross aisle, an end of an aisle, or an aisle sign, from a similar field of endeavor, Grauer discloses image processing and computer vision methods to recognize traffic signs (see Fig. 3A-3C) through classification (Grauer, [0053]).
One of ordinary skill in the art would find it obvious, prior to the applicant’s effective filing date, to combine the system of Grauer to the system of Halder as Halder already utilizes classification machine learning models to assist on classifying objects in the environment, where the two systems together are compatible.
Regarding claim 6, with all of the limitations of claim 1, the system further comprises:
a vehicle controller, wherein the vehicle controller is configured to determine a speed of the material handling vehicle, wherein the one or more signals from the perception system is based at least in part on the speed of the material handling vehicle being greater than or equal to a speed threshold.
While Halder discloses a vehicle controller, wherein the vehicle controller is configured to determine a speed of the material handling vehicle ([0038], “Vehicle systems 115 can be used to set the path and speed of autonomous vehicle 100.”), Halder does not explicitly disclose wherein one or more signals from the perception system is at least in part on the speed of the material handling vehicle being greater than or equal to a speed threshold.
However, one of ordinary skill in the art would find it obvious that Halder discloses wherein one or more signals from the perception system is at least in part on the speed of the material handling vehicle being greater than or equal to a speed threshold as setting the speed of autonomous vehicle 100 would be setting a threshold to be at least.
Regarding claim 7, with all of the limitations of claim 6, the system further comprises:
wherein the feedback operation includes generating a notification when the speed of the material handling vehicle is greater than or equal to the speed threshold, the notification including information about an operator of the material handling vehicle and information about a type of target identified from the at least one feature from the sensor data.
While Halder does not explicitly disclose generating a notification when the speed of the material handling vehicle is greater than or equal to the speed threshold, the notification including information about an operator of the material handling vehicle and information about a type of target identified from the at least one feature from the sensor data, from a similar field of endeavor, Grauer discloses an autonomous vehicle utilizing images to capture the surroundings of a vehicle. Specifically, Grauer discloses generating a notification due to undesired situations in the context of safety such as obstacles on the road or overtaking (Grauer, [0044]).
One of ordinary skill in the art would find it obvious, prior to the applicant’s effective filing date, to combine the system of Grauer to the system of Halder as generating a notification in the case of a feedback operation would allow for the user to properly understand the situation
Regarding claim 11, with all of the limitations of claim 9, the vehicle further comprises:
a feedback device in communication with the vehicle controller, wherein the feedback device is designed to provide an auditory alert, a visual alert, a tactile alert, or a combination thereof based at least in part on the vehicle controller initiating the response.
See the rationale of claim 6.
Regarding claim 12, with all of the limitations of claim 9, the vehicle further comprises:
wherein the at least one processor is further designed to:
output a deactivation signal to the vehicle controller to deactivate the response based at least in part on a change in a speed of the material handling vehicle, a change in a heading of the material handling vehicle, an increase in a distance of the material handling vehicle to the at least one target, the at least one target no longer being identifiable in the sensor data, or combinations thereof.
One of ordinary skill in the art would find it obvious to deactivate the response based as the response was to indicate to a user any undesirable condition. To continue alerting with a lack of undesirable condition may prove to be more harmful as the alerts may provide more distractions.
Regarding claim 13, with all of the limitations of claim 9, the vehicle further comprises:
wherein the response includes generating a notification when a speed of the material handling vehicle is greater than or equal to a speed threshold, the notification including information about an operator of the material handling vehicle and information about the at least one target identified.
See the rationale of claim 7.
Regarding claim 16, with all of the limitations of claim 14, the method further comprises:
determining a speed of the material handling vehicle, wherein the speed metric is associated with the speed of the material handling vehicle being greater than or equal to a speed threshold.
See the rationale of claim 6.
Regarding claim 18, with all of the limitations of claim 14, the method further comprises:
wherein performing the at least one feedback operation at the at least one feedback device further comprises:
preventing acceleration of the material handling vehicle,
activating a service brake of the material handling vehicle,
activating a regenerative brake of the material handling vehicle,
or combinations thereof.
While Halder does not explicitly disclose feedback operations, one of ordinary skill in the art would find it obvious that maintaining a certain distance (Halder, [0061]) would include preventing acceleration of the material handling vehicle when the certain distance is met to prevent changes in speed that may cause a change in the certain distance.
Regarding claim 19, with all of the limitations of claim 14, the method further comprises:
wherein the type of the target includes stop signs, cross aisles, ends of aisles, aisle signs, or other environmental features of a warehouse.
See the rationale of claim 3.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAEWOOK JUNG whose telephone number is (571)272-5470. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached on (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.J./Examiner, Art Unit 3656
/WADE MILES/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights