DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Restriction Election
Applicant’s election without traverse of Claims 11-16 in the reply filed on 04/01/2026 is acknowledged.
Claims 1-10 and 17-19 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to nonelected inventions, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 04/01/2026.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 11-16 are rejected under 35 U.S.C. 103 as being unpatentable over Smith et al. (U.S. 2019/0302764A1; Fig. 33b; ¶0099 and ¶0261-0265) in view of Vasoya et al. (US 20220001897 A1; note EFD 07/06/2020). Smith discloses “A system and method for operation of an autonomous vehicle (AV) yard truck is provided. A processor facilitates autonomous movement of the AV yard truck, and connection to and disconnection from trailers. A plurality of sensors are interconnected with the processor that sense terrain/objects and assist in automatically connecting/disconnecting trailers. A server, interconnected, wirelessly with the processor, that tracks movement of the truck around and determines locations for trailer connection and disconnection. A door station unlatches/opens rear doors of the trailer when adjacent thereto, securing them in an opened position via clamps, etc. The system computes a height of the trailer, and/or if landing gear of the trailer is on the ground and interoperates with the fifth wheel to change height” (Abstract) and “FIG. 33 is a rear-oriented perspective view of an exemplary AV yard truck and trailer hitched thereto, depicting a camera/ranging sensor combination mounted on the back of the yard truck and used to identify and track a unique feature on the front panel of the trailer; [0098] FIG. 33A is a diagram showing image processing stages used to extract tracking features in subsequent image frames during the backup maneuver of an exemplary yard truck; [0099] FIG. 33B is a diagram showing images of the back of a trailer indicating the vertical tracked feature shift in the imagery used to estimate a height differential of the trailer, and thus, the height of the fifth wheel landing gear off the ground” (¶0097-0099). See also ¶0261-0265
Regarding Claim 11, Smith discloses A method for determining a trailer lifted status of a trailer hitched to a tractor (Fig. 33, 33a, 33b; ¶0261-0265), comprising:
receiving input data including at least one of (a) images (¶0262; “The camera 3332 can be used to monitor a unique visual feature on the trailer, while the ranging sensor 3330 provides additional information “; and Fig. 33b; images 3391, 3392, etc.) from a camera mounted on the tractor (¶0262; “The camera 3332 can be used to monitor a unique visual feature on the trailer, while the ranging sensor 3330 provides additional information allowing the onboard processor system 3338 to calculate that unique feature's position in space. The determination of the height of the fifth wheel 3340 (shown in phantom) is based on the difference in the vertical position of the identified unique feature on the front panel of the trailer between the beginning and end of the hookup maneuver.), (b) a point cloud from at least one LIDAR (Fig. 33 3332/3330) mounted (Fig. 33, computer vision sensors 3332/3330 are mounted on tractor 3320) on the tractor, and (c) radar data from at least one radar (Fig. 33 3332/3330)mounted on the tractor (¶0262; “ranging sensor 3330 provides additional information”; ¶0035 “The sensor assembly can include at least one of a vision system camera, LIDAR and radar, among other known visual and spatial sensor types.”);
and processing the input data to generate the trailer lifted status indicative of whether a landing gear of the trailer as represented in the input data is lifted off a ground on which the trailer is positioned. ((Fig. 33, 33a, 33b; ¶0265; “The vision system identifies a height change (line 3395) in the tracked feature 3394 in the right frame 3392, after the fifth wheel has engaged and raised the level of the trailer front 3342. It is this height change, in which the vertical component of the position of the tracked feature 3394 allows for the computation of the elevation that the fifth wheel raises the landing gear off the ground.”)
Smith discloses that “a computer vision algorithm/process module, which can be instantiated in the processor 3338, processes data from the camera 3330” and the ranging sensor 3332 in order to determine “the elevation that the fifth wheel raises the landing gear off the ground” (¶0265).
However, Smith does not explicitly disclose that the computer vision algorithm/process module comprises processing image and range data “through a neural network” in order to determine “the elevation that the fifth wheel raises the landing gear off the ground” (¶0265).
Vasoya discloses “A vehicular trailer assist system includes a camera disposed at a rear portion of a vehicle and viewing at least rearward of the vehicle, and an electronic control unit (ECU), which includes an image processor for processing image data captured by the camera. The ECU executes a deep neural network (DNN) to determine presence of a trailer in image data captured by the camera. The DNN, responsive to processing of image data captured by the camera, determines a position of a trailer in the captured image data. The DNN, responsive to determining the position of the trailer, classifies the trailer into a trailer category. The ECU, responsive to the DNN determining the position of the trailer and based on the classification, generates an output to autonomously control the vehicle to align the vehicle with the trailer and navigate the vehicle toward the trailer.” (Abstract) and “Implementations herein include a lightweight deep neural network (DNN) based object detection method that detects and identifies various objects at or near a path of a vehicle from image data captured using a camera or imaging sensor installed on or at the vehicle. The DNN may be trained using images and/or image data (such as video images derived from captured image data) to determine presence of an object (such as a trailer) present in the captured image data and to determine a position of the determined object relative to the equipped vehicle. The DNN may be trained to identify the object in various environments (e.g., sunny, cloudy, raining) and/or situations (e.g., other objects and obstacles in the vicinity of the vehicle or determined object) and/or may be trained to determine qualities or characteristics of the determined object.” (¶0017) and “Thus, the object detection system with the trailer detection DNN provides the driver and/or other occupant (or another system of the vehicle) with a trailer hitching assist system, trailering assist system, and/or a vehicle towing automation system. The traffic light DNN provides the driver and/or driver assist system with traffic light determination, a traffic violation monitoring system and a traffic jam determination system to differentiate between actual traffic jams and traffic stoppage at traffic lights. Thus, the object detection system offers a lightweight DNN method for real time object detection for various ADAS applications.” (¶0029) “The vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ultrasonic sensors or the like. The imaging sensor or camera may capture image data for image processing and may comprise any suitable camera or sensing device” (¶0029)
Therefore Vasoya teaches: a computer vision algorithm/process module (Fig. 1, ECU 11) configured to process image and range data “through a neural network” (¶0017, DNN or CNN) in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
It would have been obvious to one with ordinary skill in the art at the time of filing of the invention to have modified the system and method for operation of an autonomous vehicle (AV) yard truck of Smith to incorporate the teachings of Vasoya to include a computer vision algorithm/process module (Fig. 1, ECU 11) configured to process image and range data through a neural network in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
Regarding Claim 12, Smith further discloses the trailer lifted status further comprising a lift metric indicative of a height of the landing gear off the ground ((Fig. 33, 33a, 33b; ¶0265; “The vision system identifies a height change (line 3395) in the tracked feature 3394 in the right frame 3392, after the fifth wheel has engaged and raised the level of the trailer front 3342. It is this height change, in which the vertical component of the position of the tracked feature 3394 allows for the computation of the elevation that the fifth wheel raises the landing gear off the ground.”)
Regarding Claim 13, Smith further discloses data that includes images, cloud points, and radar data of hitched trailers in varying states of being lifted (Fig. 33a and 33b, “If the fifth wheel 3340 is properly engaged with the trailer 3310, then the front end 3342 of the trailer will raise off the ground and the position of the tracked feature will reflect this elevation change. This is represented by the two, side-by-side image frames 3391 and 3392 in the representation 3390 of FIG. 33B. Left frame 3391 represents the image of the trailer front end 3342 before it is engaged by the fifth wheel, and thus, rests on the landing gear at a first level. This level is revealed by the corresponding level (line 3393 of the tracked feature 3394). The vision system identifies a height change (line 3395) in the tracked feature 3394 in the right frame 3392, after the fifth wheel has engaged and raised the level of the trailer front 3342.”; ¶0265)
Smith does not explicitly disclose the neural network using a trained model generated from training data that includes images, cloud points, and radar data
Vasoya teaches the neural network using a trained model generated from training data that includes images, cloud points, and radar data (¶0017; “The DNN may be trained using images and/or image data (such as video images derived from captured image data) to determine presence of an object (such as a trailer) present in the captured image data and to determine a position of the determined object relative to the equipped vehicle”) in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
It would have been obvious to one with ordinary skill in the art at the time of filing of the invention to have modified the system and method for operation of an autonomous vehicle (AV) yard truck of Smith to incorporate the teachings of Vasoya to include the neural network using a trained model generated from training data that includes images, cloud points, and radar data in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
Regarding Claim 14, Smith does not explicitly disclose further comprising: capturing a training set having at least one of a training image, a training point cloud, and a training radar data; annotating the training set to define a ground truth for operating the neural network to determine the trailer lifted status; and training the neural network using the training set to generate the trained model
Vasoya teaches further comprising: capturing a training set having at least one of a training image, a training point cloud, and a training radar data; annotating the training set to define a ground truth for operating the neural network to determine the trailer lifted status; and training the neural network using the training set to generate the trained model (¶0019-0021, Figs. 3-5; “FIGS. 3A-3C illustrate example training images 22g-22i representative of the trailer at varying distances”, “FIGS. 4A-4C illustrate image data 22j, 22k, 22m representative of the trailer in various orientations relative to the vehicle”, FIGS. 5A-5F illustrate image data 22n-22s representative of the trailer in various environmental variations”. The disclosed training data (e.g. images) includes ground truth data comprising separately verified distances, angles/orientations, and environmental variations) in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.)
It would have been obvious to one with ordinary skill in the art at the time of filing of the invention to have modified the system and method for operation of an autonomous vehicle (AV) yard truck of Smith to incorporate the teachings of Vasoya to include further comprising: capturing a training set having at least one of a training image, a training point cloud, and a training radar data; annotating the training set to define a ground truth for operating the neural network to determine the trailer lifted status; and training the neural network using the training set to generate the trained model in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
Regarding Claim 15, Smith further discloses wherein the vision system processes the input data and generates a lift metric indicative of the height above the ground of the landing gear of the trailer hitched to the tractor ((Fig. 33, 33a, 33b; ¶0265; “The vision system identifies a height change (line 3395) in the tracked feature 3394 in the right frame 3392, after the fifth wheel has engaged and raised the level of the trailer front 3342. It is this height change, in which the vertical component of the position of the tracked feature 3394 allows for the computation of the elevation that the fifth wheel raises the landing gear off the ground.”)
Smith does not explicitly disclose comprising annotating the training set to define the ground truth for a height of corresponding landing gear depicted in the training set above the ground
Vasoya discloses utilizing training image data comprising ground truth metrics and therefore teaches annotating the training set to define the ground truth (¶0019-0021, Figs. 3-5; “FIGS. 3A-3C illustrate example training images 22g-22i representative of the trailer at varying distances”, “FIGS. 4A-4C illustrate image data 22j, 22k, 22m representative of the trailer in various orientations relative to the vehicle”, FIGS. 5A-5F illustrate image data 22n-22s representative of the trailer in various environmental variations”. The disclosed training data (e.g. images) includes ground truth data comprising separately verified distances, angles/orientations, and environmental variations) in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.) in order to train the DNN to reliably identify and “to determine presence of an object present in image data captured by the camera, determine a location or position of the determined object relative to the vehicle, and optionally determine a category of the determined object.” (¶0019). Vasoya does not explicitly teach utilizing ground truth data for training the DNN to identify a trailer intended use image characteristic of for a height of corresponding landing gear depicted in the training set above the ground
“The Court quoting In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006), stated that “‘[R]ejections on obviousness cannot be sustained by mere conclusory statements; instead, there must be some articulated reasoning with some rational underpinning to support the legal conclusion of obviousness.’” KSR, 550 U.S. at ___, 82 USPQ2d at 1396. Exemplary rationales that may support a conclusion of obviousness include:
(C) Use of known technique to improve similar devices (methods, or products) in the same way;
Here, it would have been obvious to one skilled in the art at the time of the invention to include annotating the training set to define the ground truth for a height of corresponding landing gear depicted in the training set above the ground by (C) Use of known technique to improve similar devices (methods, or products) as taught by Vasoya into the teachings of Smith because it does no more than yield predictable results of utilizing ground truth data to train a Neural Network image processing system to reliably determine presence of an object present in image data captured by the camera, determine a location or position of the determined object relative to the vehicle, (¶0019) since it has been held that the combination of familiar elements according to known methods is likely to be obvious when it does no more than yield predictable results (MPEP 2143).
Regarding Claim 16, Smith does not explicitly disclose wherein the training image, the training point cloud, and the training radar data are captured from a camera, a LIDAR, and a radar positioned and oriented as on the tractor
Vasoya teaches wherein the training image, the training point cloud, and the training radar data are captured from a camera, a LIDAR, and a radar positioned and oriented as on the tractor (¶0019, “The DNN may be trained in various conditions and orientations to determine presence of an object present in image data captured by the camera, determine a location or position of the determined object relative to the vehicle, and optionally determine a category of the determined object…. the system may be trained or retrained or tuned during normal operation of the vehicle. For example, the system may perform further unsupervised learning using image data captured during normal use of the system. The system may perform further supervised learning by requesting feedback from an operator of the vehicle (e.g., requesting the operator to identify a trailer type of a trailer, etc.).” and ¶0023 “The traffic light DNN, like the trailer detection DNN, is trained on a plurality of images 122 captured by a camera 19—i.e., a camera with a field of view forward of the vehicle.” And “The vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ultrasonic sensors or the like.” ¶0029) in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
It would have been obvious to one with ordinary skill in the art at the time of filing of the invention to have modified the system and method for operation of an autonomous vehicle (AV) yard truck of Smith to incorporate the teachings of Vasoya to include wherein the training image, the training point cloud, and the training radar data are captured from a camera, a LIDAR, and a radar positioned and oriented as on the tractor in order to provide an automated vehicle trailer hitching/trailering system comprising “a lightweight DNN method for real time object detection for various ADAS applications”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Herman et al. (U.S. 2019/037498A1) discloses “Methods and apparatus are disclosed for automated detection of trailer properties. An example vehicle includes an inter-vehicle communication module and an infotainment head unit. The infotainment head unit is configured to detect presence of an attached trailer. The infotainment head unit is also configured to, in response to a determination that the attached trailer is an unrecognized trailer broadcast a request for images via the inter-vehicle communication module, perform semantic segmentation on the images, generate a three dimensional point cloud using the segmented images, and estimate a property of the attached trailer based on the three dimensional point cloud” (Abstract)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN R KIRBY whose telephone number is (571)270-3665. The examiner can normally be reached Telework: M-F, 9a-5p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lindsay Low can be reached at 571-272-1196. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN R KIRBY/Examiner, Art Unit 3747
/LINDSAY M LOW/Supervisory Patent Examiner, Art Unit 3747
Prosecution Insights