DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-4, 8, 10-11, 15-16, 18 and 20 have been amended changing the scope and contents of the claim.
Claim 21-23 have been newly added.
Claims 6 and 12-13 have been cancelled.
Applicant’s amendment filed July 31, 2026 overcomes the following objection/rejection(s) from the last Office Action of May 1, 2026:
Rejections to the claims under 35 USC § 102
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 10 and 16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 7, 10 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2021/0229509 to Hosseiny et al. (hereinafter Hosseiny), and further in view of U.S. Publication No. 2021/0070362 to Xu et al. (hereinafter Xu).
Regarding independent claim 1, Hosseiny discloses A system (abstract, “A Remote Trailer Maneuvering (RTM) system”) of obstacle detection for a trailer connected to and towed by a vehicle (paragraph 0013, “The mobile device processes the photo by identifying and characterizing environmental information for possible obstacles, such as walls, surrounding objects, etc;” paragraph 0014, “The RTM system may determine the distance using various methods, including determining a salinity level of the water, measuring relative distances between the vehicle, the trailer/trailer contents, the water, obstacles in the vicinity, and using other measurements;” paragraph 0063, “At step 720, the method may further include generating a trailer maneuver path, based on the image, from a first trailer location to the target location. This step may include identifying, based on the image, an obstacle along the trailer maneuver path, generating a modified trailer maneuver path by modifying the trailer maneuver path to avoid the obstacle”), the system comprising:
a camera positioned at a rear of the vehicle (paragraph 0054, “FIG. 6 depicts the vehicle 105 maneuvering the trailer 110 to water 610 using the RTM system 107, according to an embodiment;” paragraph 0047, “FIG. 3 depicts the example mobile device 120, configured as part of the RTM system 107, in accordance with the present disclosure. The mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1) from which to capture a photo;” Figure 6 shows the user obtaining an image of the object behind the vehicle), the camera configured to capture images of the trailer and a scene including an object (Figure 5), the camera further configured to generate image data corresponding to the scene and output the image data (paragraph 0047, “the mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1)”);
a controller on the vehicle (paragraph 0019, “FIG. 1 depicts an example computing environment 100 (also referred to herein as the trailer parking environment 100″) that can include one or more vehicle(s) 105 comprising an automotive computer 145, and a Vehicle Controls Unit (VCU) ”), the controller including an input/output interface (Figure 1, there are multiple interfaces connected to the VCU to receive and send input and outputs), a memory (paragraph 0020, “The automotive computer 145 may be or include an electronic vehicle controller, having one or more processor(s) 150 and memory 155.”), and an electronic processor (paragraph 0020, “The automotive computer 145 may be or include an electronic vehicle controller, having one or more processor(s) 150 and memory 155.”) configured to:
receive the image data from the camera (paragraph 0040, “ In an example embodiment, the user interface device 215 may receive an image of a trailer parking environment, such that the RTM system 107 can determine the target position 109 for parking the trailer 110 using the image data. ”),
analyze the object in the scene (paragraph 0051, “In order to avoid an obstacle, The RTM system 107 must characterize the visual representation of the pier from the dimensional orientation data as an obstacle using image recognition, identify the recognized object from the image 305 as an obstacle, and define relative boundaries of the obstacle(s), the vehicle 105, the trailer 110, and the target position 109 with respect to the vehicle and trailer dimensions”),
determine that the object is an obstacle using an obstacle detection algorithm (paragraph 0051, “In order to avoid an obstacle, The RTM system 107 must characterize the visual representation of the pier from the dimensional orientation data as an obstacle using image recognition, identify the recognized object from the image 305 as an obstacle, and define relative boundaries of the obstacle(s), the vehicle 105, the trailer 110, and the target position 109 with respect to the vehicle and trailer dimensions”), and
determine that the obstacle has exceeded a proximity threshold (paragraph 0064, “Identifying the obstacle in the path may include steps such as receiving a trailer profile comprising a trailer dimension determining a dimension of the obstacle based on the trailer dimension, determining a first distance from the obstacle to the trailer, determining a second distance from the obstacle to the vehicle, and generating the modified trailer maneuver path based on the first distance and the second distance;” the path is read as being maneuvered when an obstacle is within a proximity threshold of the path),
wherein in response to the determination that the obstacle has exceeded the proximity threshold, the controller controls the vehicle (paragraph 0063, “At step 720, the method may further include generating a trailer maneuver path, based on the image, from a first trailer location to the target location. This step may include identifying, based on the image, an obstacle along the trailer maneuver path, generating a modified trailer maneuver path by modifying the trailer maneuver path to avoid the obstacle;” paragraph 0064, “Identifying the obstacle in the path may include steps such as receiving a trailer profile comprising a trailer dimension determining a dimension of the obstacle based on the trailer dimension, determining a first distance from the obstacle to the trailer, determining a second distance from the obstacle to the vehicle, and generating the modified trailer maneuver path based on the first distance and the second distance;”).
Hosseiny fails to explicitly disclose as further recited. However, Xu discloses determine an articulation angle of the trailer based upon a plurality of corners of the trailer relative to the object (paragraph 0050, “In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. Further, in this embodiment, the hitch angle γ may be defined by the angle between the vehicle 14 and a line normal to a vector extending between the determined corner positions of the trailer 12. It is contemplated that, the virtual sensor system 18 may determine vehicle 14, trailer 12, and object 15 parameters in addition to those discussed herein, and it is further contemplated that, in some embodiments, the sensor system 16 may determine trailer width Tw, hitch angle γ, the position of the vehicle 14, and the position and/or path of objects 15 in the operating environment 24 of the vehicle 14 by means other than the virtual sensor system 18 described herein.” paragraph 0123, “The method 150 of operating the trailer sideswipe avoidance system 10 may further include the step 158 of determining a relationship between the object 15, the vehicle 14, and/or the trailer 12. For example, in some embodiments, the step 158 may include determining the distance robj from the object 15 to the trailer turn center O. Further in some embodiments, the step 158 may include determining whether the inner trailer boundary line 90 intersects the virtual circle 92.”),
Calculate a position of the trailer and the object relative to one another based upon the articulation angle (paragraph 0050, “In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. Further, in this embodiment, the hitch angle γ may be defined by the angle between the vehicle 14 and a line normal to a vector extending between the determined corner positions of the trailer 12. It is contemplated that, the virtual sensor system 18 may determine vehicle 14, trailer 12, and object 15 parameters in addition to those discussed herein, and it is further contemplated that, in some embodiments, the sensor system 16 may determine trailer width Tw, hitch angle γ, the position of the vehicle 14, and the position and/or path of objects 15 in the operating environment 24 of the vehicle 14 by means other than the virtual sensor system 18 described herein.” paragraph 0123, “The method 150 of operating the trailer sideswipe avoidance system 10 may further include the step 158 of determining a relationship between the object 15, the vehicle 14, and/or the trailer 12. For example, in some embodiments, the step 158 may include determining the distance robj from the object 15 to the trailer turn center O. Further in some embodiments, the step 158 may include determining whether the inner trailer boundary line 90 intersects the virtual circle 92.”),
Hosseiny is directed toward, “A Remote Trailer Maneuvering (RTM) system includes a mobile device app that identifies and processes environmental information from a photo, to identify obstacles such as walls, surrounding objects, etc (abstract).” Xu is directed toward, “A trailer sideswipe avoidance system for a vehicle towing a trailer is provided herein. The trailer sideswipe avoidance system includes a sensor system configured to detect objects in an operating environment of the vehicle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand determining corners of a trailer would allow for the ability to ensure the trailer and an obstacle do not collide. Said differently, if only a center of mass, or center of a trailer was monitored, there could easily be a collision between a trailer and an obstacle since the entire dimension of the trailer is not tracked. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to ensure the trailers dimensions are analyzed to prevent anywhere on the trailer from colliding with an obstacle.
Regarding dependent claim 4, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny further discloses the electronic processor further further configured to:
calculate a 3D position of the trailer (paragraph 0051, “The app 135 may determine the target position 109 using various dimensional orientation techniques that place a digital representation of the vehicle 105 and the trailer 110 in 3-dimensional (3D) space”), and a proximity of the object relative to the position of the trailer (paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.),”).
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny and Xu fail to explicitly disclose wherein the proximity threshold is a range of distance between the object and the trailer, the range of distance including between 5 centimeters and 1 meter.
However, Hosseiny discloses at paragraph 0066, “In another example embodiment, the generating step can include identifying, based on the image, an obstacle along the trailer maneuver path to the target location.” Thus, Hosseiny takes into account an obstacle being present along a path at all. One of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware different sensitivities can be used for sensors. For example, if the distance between a trailer and an obstacle is 100 meters, it may not be relevant to a driver at that moment. Said differently, they may only care as they get to a specific distance away (i.e. 1 meter). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to configure the sensitivity of the threshold between an obstacle and trailer based on an ideal distance for a user to be notified (i.e. early enough to not hit the obstacle, but not too early where the driver is notified before they are even within a reasonable distance).
Regarding independent claim 10, the rejection of claim 1 applies directly. Additionally, Hosseiny further discloses A method of obstacle detection for a trailer connected to and towed by a vehicle (paragraph 0059, “FIG. 7 is a flow diagram of an example method 700 for causing the RTM system 107 to maneuver the trailer 110 to the target position 109, according to the present disclosure;” paragraph 0013, “The mobile device processes the photo by identifying and characterizing environmental information for possible obstacles, such as walls, surrounding objects, etc;” paragraph 0014, “The RTM system may determine the distance using various methods, including determining a salinity level of the water, measuring relative distances between the vehicle, the trailer/trailer contents, the water, obstacles in the vicinity, and using other measurements;” paragraph 0063, “At step 720, the method may further include generating a trailer maneuver path, based on the image, from a first trailer location to the target location. This step may include identifying, based on the image, an obstacle along the trailer maneuver path, generating a modified trailer maneuver path by modifying the trailer maneuver path to avoid the obstacle”), the method comprising:
generating, by a camera positioned at a rear of the vehicle, images of the trailer and a scene including an object (paragraph 0054, “FIG. 6 depicts the vehicle 105 maneuvering the trailer 110 to water 610 using the RTM system 107, according to an embodiment;” paragraph 0047, “FIG. 3 depicts the example mobile device 120, configured as part of the RTM system 107, in accordance with the present disclosure. The mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1) from which to capture a photo;” Figure 6 shows the user obtaining an image of the object behind the vehicle; Figure 5; paragraph 0047, “the mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1)”),
outputting, by the camera, image data corresponding to the scene (paragraph 0047, “the mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1)”),
receiving, by an electronic processor, the image data (paragraph 0040, “ In an example embodiment, the user interface device 215 may receive an image of a trailer parking environment, such that the RTM system 107 can determine the target position 109 for parking the trailer 110 using the image data. ”),
analyzing, by the electronic processor, the object in the scene (paragraph 0051, “In order to avoid an obstacle, The RTM system 107 must characterize the visual representation of the pier from the dimensional orientation data as an obstacle using image recognition, identify the recognized object from the image 305 as an obstacle, and define relative boundaries of the obstacle(s), the vehicle 105, the trailer 110, and the target position 109 with respect to the vehicle and trailer dimensions”),
determining, by the electronic processor, that the object is an obstacle using an obstacle detection algorithm (paragraph 0051, “In order to avoid an obstacle, The RTM system 107 must characterize the visual representation of the pier from the dimensional orientation data as an obstacle using image recognition, identify the recognized object from the image 305 as an obstacle, and define relative boundaries of the obstacle(s), the vehicle 105, the trailer 110, and the target position 109 with respect to the vehicle and trailer dimensions”) ,
determining, by the electronic processor, that the obstacle has exceeded a proximity threshold (paragraph 0064, “Identifying the obstacle in the path may include steps such as receiving a trailer profile comprising a trailer dimension determining a dimension of the obstacle based on the trailer dimension, determining a first distance from the obstacle to the trailer, determining a second distance from the obstacle to the vehicle, and generating the modified trailer maneuver path based on the first distance and the second distance;” the path is read as being maneuvered when an obstacle is within a proximity threshold of the path), and
wherein in response to the determination that the obstacle has exceeded the proximity threshold, controlling, by the electronic processor, the vehicle (paragraph 0063, “At step 720, the method may further include generating a trailer maneuver path, based on the image, from a first trailer location to the target location. This step may include identifying, based on the image, an obstacle along the trailer maneuver path, generating a modified trailer maneuver path by modifying the trailer maneuver path to avoid the obstacle;” paragraph 0064, “Identifying the obstacle in the path may include steps such as receiving a trailer profile comprising a trailer dimension determining a dimension of the obstacle based on the trailer dimension, determining a first distance from the obstacle to the trailer, determining a second distance from the obstacle to the vehicle, and generating the modified trailer maneuver path based on the first distance and the second distance;”).
Hosseiny fails to explicitly disclose as further recited. However, Xu discloses
calculating, by the electronic processor, an articulation angle of the trailer based upon a plurality of corners of the trailer relative to the object (paragraph 0050, “In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. Further, in this embodiment, the hitch angle γ may be defined by the angle between the vehicle 14 and a line normal to a vector extending between the determined corner positions of the trailer 12. It is contemplated that, the virtual sensor system 18 may determine vehicle 14, trailer 12, and object 15 parameters in addition to those discussed herein, and it is further contemplated that, in some embodiments, the sensor system 16 may determine trailer width Tw, hitch angle γ, the position of the vehicle 14, and the position and/or path of objects 15 in the operating environment 24 of the vehicle 14 by means other than the virtual sensor system 18 described herein.” paragraph 0123, “The method 150 of operating the trailer sideswipe avoidance system 10 may further include the step 158 of determining a relationship between the object 15, the vehicle 14, and/or the trailer 12. For example, in some embodiments, the step 158 may include determining the distance robj from the object 15 to the trailer turn center O. Further in some embodiments, the step 158 may include determining whether the inner trailer boundary line 90 intersects the virtual circle 92.”);
calculating, by the electronic processor, a position of the trailer and the object relative to one another based upon the articulation angle (paragraph 0050, “In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. Further, in this embodiment, the hitch angle γ may be defined by the angle between the vehicle 14 and a line normal to a vector extending between the determined corner positions of the trailer 12. It is contemplated that, the virtual sensor system 18 may determine vehicle 14, trailer 12, and object 15 parameters in addition to those discussed herein, and it is further contemplated that, in some embodiments, the sensor system 16 may determine trailer width Tw, hitch angle γ, the position of the vehicle 14, and the position and/or path of objects 15 in the operating environment 24 of the vehicle 14 by means other than the virtual sensor system 18 described herein.” paragraph 0123, “The method 150 of operating the trailer sideswipe avoidance system 10 may further include the step 158 of determining a relationship between the object 15, the vehicle 14, and/or the trailer 12. For example, in some embodiments, the step 158 may include determining the distance robj from the object 15 to the trailer turn center O. Further in some embodiments, the step 158 may include determining whether the inner trailer boundary line 90 intersects the virtual circle 92.”),
Hosseiny is directed toward, “A Remote Trailer Maneuvering (RTM) system includes a mobile device app that identifies and processes environmental information from a photo, to identify obstacles such as walls, surrounding objects, etc (abstract).” Xu is directed toward, “A trailer sideswipe avoidance system for a vehicle towing a trailer is provided herein. The trailer sideswipe avoidance system includes a sensor system configured to detect objects in an operating environment of the vehicle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand determining corners of a trailer would allow for the ability to ensure the trailer and an obstacle do not collide. Said differently, if only a center of mass, or center of a trailer was monitored, there could easily be a collision between a trailer and an obstacle since the entire dimension of the trailer is not tracked. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to ensure the trailers dimensions are analyzed to prevent anywhere on the trailer from colliding with an obstacle.
Regarding dependent claim 22, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny discloses the electronic processor further configured to:
calculate a three-dimensional position (paragraph 0051, “a digital representation of the vehicle 105 and the trailer 110 in 3-dimensional (3D) space”)
wherein the determination of the articulation angle of the trailer is a function of the calculated three-dimensional position (paragraph 0051, “The app 135 may determine the target position 109 using various dimensional orientation techniques that place a digital representation of the vehicle 105 and the trailer 110 in 3-dimensional (3D) space, such that the RTM system 107 can identify an appropriate path for the vehicle 105 and trailer 110 to take, and determine control instructions that avoid potential obstacles (e.g., the wooden pier 325).” The path for the vehicle is read as including the articulation angle)
Further, Xu discloses the electronic processor further configured to:
calculate a (paragraph 0050, “ In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners”),
wherein the determination of the articulation angle of the trailer is a function of the calculated(paragraph 0050, “ In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. Further, in this embodiment, the hitch angle γ may be defined by the angle between the vehicle 14 and a line normal to a vector extending between the determined corner positions of the trailer 12. ”).
One of ordinary skill in the art would easily understand the trailer and obstacles exist in a 3D realm and articulation angle and dimensions can also be 3D. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Xu in order to ensure the data is as accurate as possible and in the dimension which they exist, as opposed to flat in a two dimensional realm.
Claim(s) 2-3, 11 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Hosseiny further in view of Xu as applied to claims 1 and 10 respectively above, and further in view of Bahramgiri, Mojtaba. Trailer Articulation Angle Detection and Tracking for Trailer Backup Assistant Systems. Diss. Michigan Technological University, 2021. (hereinafter Bahramgiri).
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny and Xu fail to explicitly disclose the electronic processor further configured to:
analyze the image data to determine object data points and background data points,
wherein the articulation angle of the trailer is further calculated based upon the background data points and a dimension of the trailer
However, Bahramgiri discloses the electronic processor further configured to:
analyze the image data to determine object data points and background data points (page 95, “The TAD model utilizes the radars returned point-clouds to detect the trailer and extract information on its orientation. The point-cloud processing unit in Fig. 3.5-(a) receives two sets of radars returns at each sampling instance. It should be noted that the merged point-cloud contains just a few returns associated with the trailer, and the rest of the points are associated with either other objects in the background or clutter.”),
wherein the articulation angle of the trailer is further calculated based upon the background data points and a dimension of the trailer (page 16, “These techniques include corners and edges, which can be incorporated into the hitch angle estimation model.” Page 19, “Although there are some trailers with rounded corners on their front face, trailers typically have sharp corners, which can be used as the features for detecting a trailer.” page 95, “The TAD model utilizes the radars returned point-clouds to detect the trailer and extract information on its orientation. The point-cloud processing unit in Fig. 3.5-(a) receives two sets of radars returns at each sampling instance. It should be noted that the merged point-cloud contains just a few returns associated with the trailer, and the rest of the points are associated with either other objects in the background or clutter.”)
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Bahramgiri is directed toward, “three models to detect the articulation angle between tow-vehicle and trailer using the rear-side camera and radar sensors (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Bahramgiri are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware determining if data points are objects or background can aid in trailer maneuvering. If a point is labeled background, such as the sky, it is determined to not be an object the trailer would need to maneuver around (being that it cannot hit that item). Conversely, correctly detecting points such as signs or poles as obstacles can allow a trailer to maneuver around the obstacle without causing a collision. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bahramgiri in order to ensure the scene is accurately analyzed and the system only processes relevant objects as obstacles to avoid collisions.
Regarding dependent claim 3, the rejection of claim 2 is incorporated herein. Additionally, Hosseiny, Xu and Bahramgiri in the combination fail to explicitly disclose the electronic processor further configured to:
determine that the object is an obstacle based upon the object data points and
determine that the object is not an obstacle based upon the background data points.
However, one of ordinary skill in the art would easily understand detecting the background data can aid in determination of whether or not an obstacle is present. For example, if an object appears in the sky region (i.e. background), one of ordinary skill in the art would easily understand the system would not want to determine the sky object as an obstacle, because the car cannot reasonably collide with the object in the sky. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Hosseiny, Xu and Llanos to utilize the classification of background data to inform the obstacle classification.
Regarding dependent claim 11, the rejection of claim 10 is incorporated herein. Additionally, Hosseiny and Xu fail to explicitly disclose the method further comprising analyzing, by the electronic processor, the image data to determine object data points and background data points,
wherein the articulation angle of the trailer is further calculated based upon the background data points and a dimension of the trailer.
However, Bahramgiri discloses the method further comprising analyzing, by the electronic processor, the image data to determine object data points and background data points (page 95, “The TAD model utilizes the radars returned point-clouds to detect the trailer and extract information on its orientation. The point-cloud processing unit in Fig. 3.5-(a) receives two sets of radars returns at each sampling instance. It should be noted that the merged point-cloud contains just a few returns associated with the trailer, and the rest of the points are associated with either other objects in the background or clutter.”),
wherein the articulation angle of the trailer is further calculated based upon the background data points and a dimension of the trailer (page 16, “These techniques include corners and edges, which can be incorporated into the hitch angle estimation model.” Page 19, “Although there are some trailers with rounded corners on their front face, trailers typically have sharp corners, which can be used as the features for detecting a trailer.” page 95, “The TAD model utilizes the radars returned point-clouds to detect the trailer and extract information on its orientation. The point-cloud processing unit in Fig. 3.5-(a) receives two sets of radars returns at each sampling instance. It should be noted that the merged point-cloud contains just a few returns associated with the trailer, and the rest of the points are associated with either other objects in the background or clutter.”).
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Bahramgiri is directed toward, “three models to detect the articulation angle between tow-vehicle and trailer using the rear-side camera and radar sensors (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Bahramgiri are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware determining if data points are objects or background can aid in trailer maneuvering. If a point is labeled background, such as the sky, it is determined to not be an object the trailer would need to maneuver around (being that it cannot hit that item). Conversely, correctly detecting points such as signs or poles as obstacles can allow a trailer to maneuver around the obstacle without causing a collision. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bahramgiri in order to ensure the scene is accurately analyzed and the system only processes relevant objects as obstacles to avoid collisions.
Regarding dependent claim 21, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny and Xu fail to explicitly disclose wherein the articulation angle of the trailer is an instantaneous articulation angle.
However, Bahramgiri discloses wherein the articulation angle of the trailer is an instantaneous articulation angle (page 10, “The employed MedianFlow tracking tech nique enables the model to improve the processing time and demonstrate satisfactory performance for real-time hitch angle estimation tasks”).
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Bahramgiri is directed toward, “three models to detect the articulation angle between tow-vehicle and trailer using the rear-side camera and radar sensors (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Bahramgiri are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand providing instantaneous analysis and feedback to a driving user could prevent collisions. Said differently, if the data is not processed fast enough an obstacle may be hit. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bahramgiri in order to ensure the information is provided to the system as quickly as possible in order to ensure obstacles are avoided.
Claim(s) 9, 15 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Hosseiny and Xu as applied to claims 1 and 10 respectively above, and further in view of U.S. Publication No. 2022/0161853 to Llanos et al. (hereinafter Llanos).
Regarding dependent claim 9, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny further discloses wherein the obstacle detection algorithm includes producing a coordinate plot of the image data including object data points (paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.);” paragraph 0053, “In one example embodiment, The RTM system 107 may determine reference points to determine vehicle, trailer/trailer payload, and obstacle boundaries.”)
Hosseiny and Xu fails to explicitly disclose as further recited. However, Llanos discloses wherein the obstacle detection algorithm includes producing a coordinate plot of the image data including object data points and background data points (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background)).
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Llanos is directed toward “A method for autonomously parking a vehicle-trailer system is provided (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Llanos are directed toward similar methods of endeavor of trailer maneuvering and parking. Further, one of ordinary skill in the art before the effective filing date would easily be aware detecting both foreground and background data can be helpful when maneuvering the trailer. Said differently, understanding background data allows for a better understanding of the scene as a whole. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Llanos in order to ensure an accurate image is processed and understood based on both object data and background data.
Regarding dependent claim 15, the rejection of claim 10 is incorporated herein. Additionally, Hosseiny further discloses the method further comprising generating by the electronic processor, a coordinate plot of the image data including object data points (paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.);” paragraph 0053, “In one example embodiment, The RTM system 107 may determine reference points to determine vehicle, trailer/trailer payload, and obstacle boundaries.”)
Hosseiny and Xu fails to explicitly disclose as further recited. However, Llanos discloses the method further comprising generating by the electronic processor, a coordinate plot of the image data including object data points and background data points (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background)).
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Llanos is directed toward “A method for autonomously parking a vehicle-trailer system is provided (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Llanos are directed toward similar methods of endeavor of trailer maneuvering and parking. Further, one of ordinary skill in the art before the effective filing date would easily be aware detecting both foreground and background data can be helpful when maneuvering the trailer. Said differently, understanding background data allows for a better understanding of the scene as a whole. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Llanos in order to ensure an accurate image is processed and understood based on both object data and background data.
Regarding dependent claim 23, the rejection of claim 1 is incorporated herein. Additionally, Xu in the combination further discloses calculate a position of the plurality of corners of the trailer using the object data points and the background data points (paragraph 0050, “ In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners.” determining the corner points means that other points were excluded (background points)); and
determine the articulation angle using the position of the plurality of corners of the trailer relative to the object ( paragraph 0012, “ determine whether an object detected in the operating environment of the vehicle is in the travel path of the towed trailer,” paragraph 0050, “ In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. ” paragraph 0019, “the angle between a line extending from the trailer turn center to the object ”).
Hosseiny and Xu fail to explicitly disclose as further recited. However, Llanos discloses generate object data points and background data points from the image data (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background));
generate a coordinate plot using the object data points and the background data points (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background));
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Llanos is directed toward “A method for autonomously parking a vehicle-trailer system is provided (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Llanos are directed toward similar methods of endeavor of trailer maneuvering and parking. Further, one of ordinary skill in the art before the effective filing date would easily be aware detecting both foreground and background data can be helpful when maneuvering the trailer. Said differently, understanding background data allows for a better understanding of the scene as a whole. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Llanos in order to ensure an accurate image is processed and understood based on both object data and background data.
Claim(s) 5, 8 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Hosseiny and Xu as applied to claims 1 and 10 respectively above, and further in view of U.S. Patent No. 6,879,914 to Hoenes et al. (hereinafter Hoenes).
Regarding dependent claim 5, the rejection of claim 1 is incorporated herein. Additionally, Hosseiny and Xu fails to explicitly disclose wherein the proximity threshold is a distance of less than 1 meter, and wherein in response to the determination that the obstacle has exceeded the proximity threshold, the controller stops the vehicle
However, Hoenes discloses wherein the proximity threshold is a distance of less than 1 meter (column 3, line 39, “the distance to an obstacle may also be output by a digital display of the distance value in meters, centimeters or another suitable unit of length;” column 4, line 11, “output a warning when a front side 21 of spare wheel 3 has approached obstacle 5 up to minimum distance 19.”), and wherein in response to the determination that the obstacle has exceeded the proximity threshold, the controller stops the vehicle (column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19. ”)
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Hoenes is directed toward “A distance-measuring device is used for measuring a distance between a vehicle and an obstacle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Hoenes are directed toward similar methods of endeavor of autonomous vehicle configurations. Further, one of ordinary skill in the art before the effective filing date would easily be aware when detecting obstacles as related to autonomous driving, obstacles at specific distances can be of varying importance. Said differently, closer obstacles may be immediately urgent for a user to be aware, while as further away obstacles may not be as relevant. For safety, close obstacles may have a more immediate impact, and thus a user may want a vehicle to stop if one is close, however if an obstacle is further aware, a user may only want the vehicle to alert a driver so the driver could still have time to react themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Hoenes in order to ensure vehicle driving systems are cognizant of distance of obstacles of most importance to users while not constantly stopping a vehicle based on obstacles not immediately close to cause collisions.
Regarding dependent claim 8, the rejection of claim 1 is incorporated herein. Additionally, Xu discloses wherein the electronic processor determines that the obstacle has exceeded the first proximity threshold and the second proximity threshold by comparing one of the plurality of corners of the trailer to the obstacle (paragraph 0050, “In some embodiments, in which the position of the corners of the trailer 12 are determined;” paragraph 0053, “More specifically, the object proximity sensor 17 may provide the trailer sideswipe avoidance system 10 with proximity information of the object 15, which may include information estimating a location of the object 15 or objects 15 relative to the vehicle 14 and/or trailer 12. ”… “The object proximity sensor 17 may include an individual sensor, multiple sensors, and various combinations of sensors and sensor systems to capture, generate, and output information characterizing the proximity of the object 15 adjacent to the vehicle 14 and/or trailer 12, as described in more detail herein.” characterizing proximity is read as comparing to thresholds; further Xu uses thresholds of time to determine how close the trailer is to a collision (paragraph 0113)).
Hosseiny and Xu fails to explicitly disclose as further recited. However, Hoenes discloses the electronic processor is further configured to:
determine that the obstacle has exceeded a first proximity threshold and a second proximity threshold, the second proximity threshold being a closer proximity between the trailer and the obstacle (column 6, line 50, “ Corresponding to the warning threshold shown in FIG. 3 and predefined by minimum distances 47, 48, in each case further warning thresholds may be provided, preferably parallel to minimum distances 47, 48 with greater distance to the vehicle, which, as a function of distance, warn the driver, in accordance with the distance allocated to the specific warning threshold, of an obstacle coming closer to the vehicle.”),
generate an alert in response to the obstacle exceeding the first proximity threshold (column 4, line 11, “output a warning when a front side 21 of spare wheel 3 has approached obstacle 5 up to minimum distance 19.”), and
control the vehicle in response to the obstacle exceeding the second proximity threshold (column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19. ”),
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Hoenes is directed toward “A distance-measuring device is used for measuring a distance between a vehicle and an obstacle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Hoenes are directed toward similar methods of endeavor of autonomous vehicle configurations. Further, one of ordinary skill in the art before the effective filing date would easily be aware when detecting obstacles as related to autonomous driving, obstacles at specific distances can be of varying importance. Said differently, closer obstacles may be immediately urgent for a user to be aware, while as further away obstacles may not be as relevant. For safety, close obstacles may have a more immediate impact, and thus a user may want a vehicle to stop if one is close, however if an obstacle is further aware, a user may only want the vehicle to alert a driver so the driver could still have time to react themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Hoenes in order to ensure vehicle driving systems are cognizant of distance of obstacles of most importance to users while not constantly stopping a vehicle based on obstacles not immediately close to cause collisions.
Regarding dependent claim 14, the rejection of claim 10 is incorporated herein. Additionally, Hosseiny and Xu fails to explicitly disclose the method further comprising:
determining, by the electronic processor, that the obstacle has exceeded a first proximity threshold and a second proximity threshold, the second proximity threshold being a closer proximity between the trailer and the obstacle,
generating, by the electronic processor, an alert in response to the obstacle exceeding the first proximity threshold, and
controlling, by the electronic processor, the vehicle in response to the obstacle exceeding the second proximity threshold.
However, Hoenes discloses the method further comprising:
determining, by the electronic processor, that the obstacle has exceeded a first proximity threshold and a second proximity threshold, the second proximity threshold being a closer proximity between the trailer and the obstacle (column 6, line 50, “ Corresponding to the warning threshold shown in FIG. 3 and predefined by minimum distances 47, 48, in each case further warning thresholds may be provided, preferably parallel to minimum distances 47, 48 with greater distance to the vehicle, which, as a function of distance, warn the driver, in accordance with the distance allocated to the specific warning threshold, of an obstacle coming closer to the vehicle.”),
generating, by the electronic processor, an alert in response to the obstacle exceeding the first proximity threshold (column 4, line 11, “output a warning when a front side 21 of spare wheel 3 has approached obstacle 5 up to minimum distance 19.”), and
controlling, by the electronic processor, the vehicle in response to the obstacle exceeding the second proximity threshold (column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19. ”).
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Hoenes is directed toward “A distance-measuring device is used for measuring a distance between a vehicle and an obstacle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Hoenes are directed toward similar methods of endeavor of autonomous vehicle configurations. Further, one of ordinary skill in the art before the effective filing date would easily be aware when detecting obstacles as related to autonomous driving, obstacles at specific distances can be of varying importance. Said differently, closer obstacles may be immediately urgent for a user to be aware, while as further away obstacles may not be as relevant. For safety, close obstacles may have a more immediate impact, and thus a user may want a vehicle to stop if one is close, however if an obstacle is further aware, a user may only want the vehicle to alert a driver so the driver could still have time to react themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Hoenes in order to ensure vehicle driving systems are cognizant of distance of obstacles of most importance to users while not constantly stopping a vehicle based on obstacles not immediately close to cause collisions.
Claim(s) 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Hosseiny, and further in view of Xu, Llanos, Bahramgiri and Hoenes.
Regarding independent claim 16, Hosseiny discloses A system of obstacle detection for a trailer connected to and towed by a vehicle (abstract, “A Remote Trailer Maneuvering (RTM) system;” paragraph 0002, “Trailer backup assist systems for vehicles may include an onboard user interface that allows the user to steer a trailer towed by the vehicle through an automated steering controller that provides the steering motion that moves the trailer along a user-defined path curvature.”), the system comprising:
a camera positioned at a rear of the vehicle, the camera configured to capture images of the trailer and a scene including an object, the camera further configured to generate image data corresponding to the scene and output the image data paragraph 0054, “FIG. 6 depicts the vehicle 105 maneuvering the trailer 110 to water 610 using the RTM system 107, according to an embodiment;” paragraph 0047, “FIG. 3 depicts the example mobile device 120, configured as part of the RTM system 107, in accordance with the present disclosure. The mobile device 120 is shown with an interface 345 that in a photo mode outputs an image of the example trailer parking environment 100 (shown from FIG. 1) from which to capture a photo;” Figure 6 shows the user obtaining an image of the object behind the vehicle);
a controller on the vehicle (paragraph 0019, “FIG. 1 depicts an example computing environment 100 (also referred to herein as the trailer parking environment 100″) that can include one or more vehicle(s) 105 comprising an automotive computer 145, and a Vehicle Controls Unit (VCU) ”), the controller including an input/output interface (Figure 1, there are multiple interfaces connected to the VCU to receive and send input and outputs), a memory (paragraph 0020, “The automotive computer 145 may be or include an electronic vehicle controller, having one or more processor(s) 150 and memory 155.”), and an electronic processor (paragraph 0020, “The automotive computer 145 may be or include an electronic vehicle controller, having one or more processor(s) 150 and memory 155.”) configured to:
receive the image data from the camera (paragraph 0040, “ In an example embodiment, the user interface device 215 may receive an image of a trailer parking environment, such that the RTM system 107 can determine the target position 109 for parking the trailer 110 using the image data. ”),
generate object data points (paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.);”)
generate a coordinate plot of the image data including object data points (paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.);” paragraph 0053, “In one example embodiment, The RTM system 107 may determine reference points to determine vehicle, trailer/trailer payload, and obstacle boundaries.”)
calculate a 3D position of the trailer relative to the object using the instantaneous articulation angle (paragraph 0014, “provide other information useful for completion of the trailer maneuvering operation;” the trailer angle/position is read as other information useful for the trailer maneuvering in that the system needs to know the location (thus the angle) of the trailer; paragraph 0051, “The app 135 may determine the target position 109 using various dimensional orientation techniques that place a digital representation of the vehicle 105 and the trailer 110 in 3-dimensional (3D) space;” paragraph 0055, “the RTM system 107 may send information 132 to the remote device 120 that includes, for example, a…graphic information indicative of obstacles and their location (e.g., dimension and location characteristics associated with potential obstacles such as trees, bushes, etc.),”),
determine that the object is an obstacle using an obstacle detection algorithm (paragraph 0051, “In order to avoid an obstacle, The RTM system 107 must characterize the visual representation of the pier from the dimensional orientation data as an obstacle using image recognition, identify the recognized object from the image 305 as an obstacle, and define relative boundaries of the obstacle(s), the vehicle 105, the trailer 110, and the target position 109 with respect to the vehicle and trailer dimensions”), and
Hosseiny fails to explicitly disclose as further recited. However, Xu discloses
calculate a position of a plurality of corners of the trailer using the object data points and background data points (paragraph 0050, “ In some embodiments, in which the position of the corners of the trailer 12 are determined, trailer width Tw may be defined by the Euclidean distance between the determined corners. ” determining the corner points means that other points were excluded (background points)),
Hosseiny is directed toward, “A Remote Trailer Maneuvering (RTM) system includes a mobile device app that identifies and processes environmental information from a photo, to identify obstacles such as walls, surrounding objects, etc (abstract).” Xu is directed toward, “A trailer sideswipe avoidance system for a vehicle towing a trailer is provided herein. The trailer sideswipe avoidance system includes a sensor system configured to detect objects in an operating environment of the vehicle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand determining corners of a trailer would allow for the ability to ensure the trailer and an obstacle do not collide. Said differently, if only a center of mass, or center of a trailer was monitored, there could easily be a collision between a trailer and an obstacle since the entire dimension of the trailer is not tracked. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to ensure the trailers dimensions are analyzed to prevent anywhere on the trailer from colliding with an obstacle.
Hosseiny and Xu fail to explicitly disclose as further recited. However, Llanos discloses generate object data points and background data points from the image data (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background)),
generate a coordinate plot of the image data including object data points and background data points (paragraph 0032, “The scene understanding module 170 receives the images 123 a (i.e., raw images) from the camera(s) 122 and assigns labels to each pixel of the image 123 a; the image labels may include, but are not limited to, obstacle, free parking spot, drivable road, grass, building, sidewalk, tree, fence, unknown solid;” as seen in Figure 3, unknown solid represents sky (i.e. background)),
As noted above, Hosseiny and Xu are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Llanos is directed toward “A method for autonomously parking a vehicle-trailer system is provided (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu and Llanos are directed toward similar methods of endeavor of trailer maneuvering and parking. Further, one of ordinary skill in the art before the effective filing date would easily be aware detecting both foreground and background data can be helpful when maneuvering the trailer. Said differently, understanding background data allows for a better understanding of the scene as a whole. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Llanos in order to ensure an accurate image is processed and understood based on both object data and background data.
Hosseiny, Xu and Llanos in the combination fail to explicitly disclose as further recited. However, Bahramgiri discloses calculate an instantaneous articulation angle of the trailer based upon the plurality of corners of the trailer relative to the object (page 10, “The employed MedianFlow tracking technique enables the model to improve the processing time and demonstrate satisfactory performance for real-time hitch angle estimation tasks”)
As noted above, Hosseiny, Xu and Llanos are directed toward similar methods of endeavor of trailer maneuvering and parking. Further, Bahramgiri is directed toward, “three models to detect the articulation angle between tow-vehicle and trailer using the rear-side camera and radar sensors (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu, Llanos and Bahramgiri are directed toward similar methods of endeavor of trailer analysis while operating a vehicle. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware determining if data points are objects or background can aid in trailer maneuvering. If a point is labeled background, such as the sky, it is determined to not be an object the trailer would need to maneuver around (being that it cannot hit that item). Conversely, correctly detecting points such as signs or poles as obstacles can allow a trailer to maneuver around the obstacle without causing a collision. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bahramgiri in order to ensure the scene is accurately analyzed and the system only processes relevant objects as obstacles to avoid collisions.
Hosseiny, Xu, Llanos and Bahramgiri in the combination fail to explicitly disclose as further recited. However, Hoenes discloses determine that the obstacle has exceeded a first proximity threshold and a second proximity threshold, the second proximity threshold being a closer proximity between the trailer and the obstacle (column 6, line 50, “ Corresponding to the warning threshold shown in FIG. 3 and predefined by minimum distances 47, 48, in each case further warning thresholds may be provided, preferably parallel to minimum distances 47, 48 with greater distance to the vehicle, which, as a function of distance, warn the driver, in accordance with the distance allocated to the specific warning threshold, of an obstacle coming closer to the vehicle.”),
wherein in response to the obstacle exceeding the first proximity threshold, the controller generates an alert (column 4, line 11, “output a warning when a front side 21 of spare wheel 3 has approached obstacle 5 up to minimum distance 19.”), and
wherein in response to the obstacle exceeding the second proximity threshold, the controller controls the vehicle (column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19. ”).
As can be seen above, Hosseiny, Xu, Llanos and Bahramgiri are directed toward similar methods of endeavor of vehicle trailer driving systems. Additionally, Hosseiny is directed toward “The RTM controller maneuvers the trailer, using an autonomous vehicle controller, to the target position using the control instructions (abstract).” Hoenes is directed toward “A distance-measuring device is used for measuring a distance between a vehicle and an obstacle (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Hosseiny, Xu, Llanos, Bahramgiri and Hoenes are directed toward similar methods of endeavor of autonomous vehicle configurations. Further, one of ordinary skill in the art before the effective filing date would easily be aware when detecting obstacles as related to autonomous driving, obstacles at specific distances can be of varying importance. Said differently, closer obstacles may be immediately urgent for a user to be aware, while as further away obstacles may not be as relevant. For safety, close obstacles may have a more immediate impact, and thus a user may want a vehicle to stop if one is close, however if an obstacle is further aware, a user may only want the vehicle to alert a driver so the driver could still have time to react themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Hoenes in order to ensure vehicle driving systems are cognizant of distance of obstacles of most importance to users while not constantly stopping a vehicle based on obstacles not immediately close to cause collisions.
Regarding dependent claim 17, the rejection of claim 16 is incorporated herein. Additionally, Hosseiny, Xu, Llanos, Bahramgiri and Hoenes in the combination fail to explicitly disclose wherein the electronic processor determines that the object is an obstacle based upon the object data points and determines that the object is not an obstacle based upon the background data points.
However, one of ordinary skill in the art would easily understand detecting the background data can aid in determination of whether or not an obstacle is present. For example, if an object appears in the sky region (i.e. background), one of ordinary skill in the art would easily understand the system would not want to determine the sky object as an obstacle, because the car cannot reasonably collide with the object in the sky. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Hosseiny, Xu, Llanos, Bahramgiri and Hoenes to utilize the classification of background data to inform the obstacle classification.
Regarding dependent claim 18, the rejection of claim 16 is incorporated herein. Additionally, Bahramgiri in the combination further discloses wherein the electronic processor further calculates the instantaneous articulation angle of the trailer relative to the object based upon a dimension of the trailer and the plurality of the corners of the trailer (page 16, “These techniques include corners and edges, which can be incorporated into the hitch angle estimation model.” Page 19, “Although there are some trailers with rounded corners on their front face, trailers typically have sharp corners, which can be used as the features for detecting a trailer.” page 95, “The TAD model utilizes the radars returned point-clouds to detect the trailer and extract information on its orientation. The point-cloud processing unit in Fig. 3.5-(a) receives two sets of radars returns at each sampling instance. It should be noted that the merged point-cloud contains just a few returns associated with the trailer, and the rest of the points are associated with either other objects in the background or clutter.”)
One of ordinary skill in the art before the effective filing date would easily be aware when performing trailer maneuver analysis, the dimensions of the trailer are of utmost importance. Further, the dimensions allow for an understanding of the corners, which are represent the widest parts of the trailer that could collide with an obstacle. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Bahramgiri in order to ensure the dimensions of the specific trailer in question are understood, to ensure the trailer does not collide with an object in its path.
Regarding dependent claim 19, the rejection of claim 16 is incorporated herein. However, Hosseiny, Xu, Llanos, Bahramgiri and Hoenes in the combination as a whole fail to explicitly discloses wherein the first proximity threshold is a range of distance between the object and the trailer, the range of distance including between 5 centimeters and 1 meter but greater than the second proximity threshold, and
wherein the second proximity threshold is a range of distance between the object and the trailer, the range of distance including between 5 centimeters and 1 meter but less than the first proximity threshold.
However, Hosseiny discloses at paragraph 0066, “In another example embodiment, the generating step can include identifying, based on the image, an obstacle along the trailer maneuver path to the target location.” Thus, Hosseiny takes into account an obstacle being present along a path at all. One of ordinary skill in the art before the effective filing date of the claimed invention would easily be aware different sensitivities can be used for sensors. For example, if the distance between a trailer and an obstacle is 100 meters, it may not be relevant to a driver at that moment. Said differently, they may only care as they get to a specific distance away (i.e. 1 meter). Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date to configure the sensitivity of the threshold between an obstacle and trailer based on an ideal distance for a user to be notified (i.e. early enough to not hit the obstacle, but not too early where the driver is notified before they are even within a reasonable distance).
Additionally, as seen in Hoenes Figure 1, there are a variety of distance thresholds (elements 6, 19 and 20) to which the system references to determine relevant alerts or controls. Hoenes describes at column 4, line 38, “Therefore, in the position shown in FIG. 1, the motor vehicle has already reached minimum distance 19 to house wall 5. Already in the position shown, the warning that the minimum distance has been reached is output via display unit 12 or loudspeaker 14 to a driver of motor vehicle 1” and column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19.” Thus, Hoenes clearly discloses the principal of multiple different distance thresholds coordinating to a response of the system (i.e. alert and stopping the vehicle). One of ordinary skill in the art before the effective filing date of the claimed invention would easily understand the distances can be adjusted based on how early/late to have the system perform the action relative to the obstacle. Thus, one of ordinary skill in the art before the effective filing date of the claimed invention would easily be able to modify the teaching of Hosseiny, Xu, Llanos, Bahramgiri and Hoenes so that the threshold distances can be altered based on how early a person should be warned or the vehicle controlled.
Regarding dependent claim 20, the rejection of claim 16 is incorporated herein. Additionally, Hoenes discloses wherein in response to the determination that the obstacle has exceeded the first proximity threshold, the controller generates an alert for a driver of the vehicle, the alert indicating that the trailer and the obstacle are in danger of colliding (column 4, line 11, “output a warning when a front side 21 of spare wheel 3 has approached obstacle 5 up to minimum distance 19;” approaching the obstacle is read as being in danger of colliding), and
wherein in response to the determination that the obstacle has exceeded the second proximity threshold, the controller stops the vehicle (column 4, line 14, “An indication of the minimum distance to the obstacle or a prompting to stop the vehicle is therefore already output when the bumper of vehicle rear 2 has reached second distance 20 which is indeed less than first distance 6, but greater than minimum distance 19. ”).
One of ordinary skill in the art before the effective filing date would easily be aware when detecting obstacles as related to autonomous driving, obstacles at specific distances can be of varying importance. Said differently, closer obstacles may be immediately urgent for a user to be aware, while as further away obstacles may not be as relevant. For safety, close obstacles may have a more immediate impact, and thus a user may want a vehicle to stop if one is close, however if an obstacle is further aware, a user may only want the vehicle to alert a driver so the driver could still have time to react themselves. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Hoenes in order to ensure vehicle driving systems are cognizant of distance of obstacles of most importance to users while not constantly stopping a vehicle based on obstacles not immediately close to cause collisions.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights