Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This is the second Office Action on the merits. Claims 1-20 are currently pending. Claims 1, 10, and 16 are currently amended. This action is final.
Response to Amendment
The amendments filed on 01/30/2026 have been entered.
In view of the Claims, Applicant’s amendments have been acknowledged.
Response to Arguments
Applicant’s arguments, see page 7-8, filed 01/30/2026, with respect to the rejection(s) of claims 1-20 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Mahnken et al. (US20210086695A1) in view of Tanaka (US2023316539A1), and further in view of Hu et al. (US20230316772A1), hereinafter Mahnken, Tanaka, and Hu, respectively.
In regards to the 35 USC 103 rejections, Applicant argues that the prior arts of record do not teach or suggest the amended limitation. Particularly, generating a real-time virtual bottom view of an object that is beneath the vehicle and outside the current field of view of all vehicle cameras using earlier-captured image data. Examiner found argument persuasive. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Mahnken in view of Tanaka, and further in view of Hu. Each of the dependent claims, depend directly or indirectly from the independent claims 1, 10, and 16, and by dependency of the independent claims are rejected under 35 USC 103, as discussed below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-7, 10, 12-14, and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Mahnken in view of Tanaka, and further in view of Hu.
Regarding claim 1, Mahnken teaches of a system for generating and displaying a virtual bottom view associated with a vehicle ("systems…for generating and presenting underbody view of a vehicle", [0001]), the system comprising: one or more movement sensors ("The chassis sensor may be an IMU, an inclinometer, a steering angle sensor, a wheel speed sensor", [0065]); and a processor ("a processor", [0006]) programmed to: generate, during a first time period, first image data associated with a first region of a parking zone external to the vehicle ("The methods is next operative to capture 330 an image of the FOV using a camera wherein the FOV includes the off-road surface", [0051], implicit that a parking zone falls within an off-road surface), wherein the first region of the parking zone includes an object ("objects within the image, such as rock edges, obstructions, or the like", [0051]); generate, via the one or more movement sensors, movement data associated with movement of the vehicle ("The processor 240 is further operative to receive vehicle dynamics data, such as steering angle, IMU 233 data, wheel speed, suspension data and inclinometer data to generate a kinematic model of the vehicle", [0046]) while the first image data is generated ("The processor 240 is then operative to coordinate the kinematic model of the vehicle with the three-dimensional model of the terrain and to generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046]), implicit that movement data is generated while or after the image data has already been generated); execute a synthesis model to synthesize the view with the three- dimensional view ("A composited three-dimensional representation may be generated in response to a combination of the image and the LiDAR depth map", [0064], "Images of the FOV captured by the camera 220 may be correlated with the three-dimensional depth map to generate an augmented image or a virtual three-dimensional representation of the vehicle proximate area", [0041]); and generate and display the virtual bottom view on a vehicle display, wherein the virtual bottom view is generated based on the synthesized view and the three-dimensional view, enabling a user to see a three-dimensional virtual view of the object when the object is not in the current field of view ("generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046], "…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, Mahnken does not teach of a plurality of cameras; generating, via the plurality of cameras, image data; execute a Structure from Motion (SfM) model based on the first image data generated during the first time period and the associated movement data, wherein the execution of the SfM model generates a three-dimensional view associated with the object; generate, via the plurality of cameras and during a second time period subsequent the first time period, a real-time view of a second region of the parking zone within a current field of view of the plurality of cameras, wherein the object is not in a current field of view of any camera of the vehicle including the plurality of cameras during the second time period and is located beneath the vehicle; during the second time period in which the object is not in the current field of view; and during a parking event and is located beneath the vehicle.
Tanaka, in the same field of endeavor, teaches of a plurality of cameras ("four panoramic cameras 12 may be mounted on the front end, rear end, and left and right sides of the vehicle 2 so that all of the surroundings of the vehicle 2 can be captured", [0029]); generating, via the plurality of cameras, image data ("four panoramic cameras 12 may be mounted on the front end, rear end, and left and right sides of the vehicle 2 so that all of the surroundings of the vehicle 2 can be captured", [0029]); execute a Structure from Motion (SfM) model based on the first image data generated during the first time period and the associated movement data, wherein the execution of the SfM model generates a three-dimensional view associated with the object ("In the section of the road in which the vehicle 2 is traveling, the position estimation unit 44 of the present embodiment estimates the positions of the vehicle 2 at the times of generation of respective images and the positions of an individual detected feature relative to these positions in accordance with the technique of "structure from motion (SfM)". More specifically, the position estimation unit 44 estimates the positional relationship between a feature of interest and the vehicle 2, based on the position of the feature of interest in each of two or more images obtained at different timings during travel of the vehicle 2 and the distance traveled by the vehicle 2 between the timings of generation of the images", [0061], the generating of a 3D view of an object is implied with the use of SfM); generate, via the plurality of cameras and during a second time period subsequent the first time period, a real-time view of a second region of the parking zone within a current field of view of the plurality of cameras ("detect one or more predetermined features from a first image representing a first area around the vehicle in a section of the road in which the vehicle is traveling…and detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006], "while" implies real-time); during the second time period in which the object is not in the current field of view ("…detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006], "This enables the feature detection device to detect a feature that cannot be seen from the first image capturing unit because of, for example, an obstacle, from the second image", [0078]); and during a parking event ("a feature is detected from a panoramic image at the time of parking the vehicle", [0068]).
However, Tanaka does not teach of wherein the object is not in a current field of view of any camera of the vehicle including the plurality of cameras during the second time period and is located beneath the vehicle; and when the object is located beneath the vehicle.
Hu, in the same field of endeavor, teaches of wherein the object is not in a current field of view of any camera of the vehicle including the plurality of cameras during the second time period and is located beneath the vehicle ("existing vehicle Surround View Systems can only visualize front, left, rear, and right sides of a vehicle, leaving a significant blind spot: the area under the vehicle, which may be multiple square meters or more", [0008], "cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area", [0014], implicit that the object is not in the field of view of any camera of the vehicle since the image data is cached to later reconstruct the area under the vehicle); and when the object is located beneath the vehicle ("virtually reconstruct the area under the vehicle", [0088]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the method for generating and displaying the bottom view of a vehicle of Mahnken with the teaching of Tanaka to utilize a Structure from Motion (SfM) model and real-time view of the second region and a plurality of cameras during parking and the teaching of Hu to specify the object not in any camera field of view and underneath the vehicle with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the accuracy of real-space position of the objects (Tanaka, [0061]) and allows the system to overcome blind spots during parking and maneuvering (Tanaka, [0078]; Hu, [0236]).
Regarding claim 2, modified Mahnken teaches of all limitations of claim 1 as stated above, additionally, wherein the execution of the synthesis model further synthesizes the view generated during the second time period with the first image data generated during the first time period ("A composite three-dimensional representation may be generated in response to a combination or the image and the LiDAR depth map", [0064], "The images may be stored in memory 450", [0060], stored in memory implies that image data has occurred in the past, or the "first time period", "the augmented image is generated in response to the host vehicle being located over the off-road surface", [0036]).
However, modified Mahnken does not explicitly teach of a real-time view.
Tanaka, in the same field of endeavor, teaches of a real-time view ("…detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006], "while" implies real-time).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the synthesis model of modified Mahnken with the real-time view of Tanaka with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the accuracy of the system by maintaining up to date information that is most representative of the real-space positions of the features within view (Tanaka, [0008]).
Regarding claim 3, modified Mahnken teaches of all limitations of claim 1 as stated above, additionally, wherein the processor ("a processor", [0006]) is further programmed to: generate a transparent image of the vehicle on the virtual bottom view, wherein the first region of the parking zone is displayed beneath the transparent image ("The exemplary embodiment is operative to generate a three-dimensional terrain map of the off-road surface 120 relative to the critical underbody components of the vehicle…This "invisible underbody" map provide the vehicle operator a view of the underbody of the off-road vehicle", [0036]-[0037]).
Regarding claim 4, modified Mahnken teaches of all limitations of claim 3 as stated above, additionally, wherein the three-dimensional view associated with the object is displayed beneath the transparent image ("The three-dimensional terrain model and the vehicle pose estimation may then be used to generate an augmented image wherein the augmented image may include a graphical representation of a vehicle suspension system, vehicle driveline, and wheels overlaid on to an image of the underbody terrain", [0057]).
Regarding claim 5, modified Mahnken teaches of all limitations of claim 1 as stated above, additionally, wherein the virtual bottom view is a bird-eye-view of the parking zone ("Various image views may include a bird's eye view, vehicle side profile, or driver's view with invisible engine hood", [0036]).
Regarding claim 6, modified Mahnken teaches of all limitations of claim 1 as stated above, additionally, wherein the virtual bottom view is a perspective view of the parking zone ("Various image views may include a bird's eye view, vehicle side profile, or driver's view with invisible engine hood", [0036]).
Regarding claim 7, modified Mahnken teaches of all limitations of claim 1 as stated above, additionally, wherein the processor ("a processor", [0006]) is further programmed to: execute a semantic segmentation model on the first image data to categorize a portion of the first image data as the object ("Image processing technique may be used to identify and locate objects within the FOV", [0039]).
However, modified Mahnken does not teach of wherein the SfM model is executed on the portion of the first image data.
Tanaka, in the same field of endeavor, teaches of wherein the SfM model is executed on the portion of the first image data ("In the section of the road in which the vehicle 2 is traveling, the position estimation unit 44 of the present embodiment estimates the positions of the vehicle 2 at the times of generation of respective images and the positions of an individual detected feature relative to these positions in accordance with the technique of "structure from motion (SfM)".", [0061]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the semantic segmentation model to categorize objects of modified Mahnken with the teachings of Tanaka to execute an SfM model on image data with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to accurately determine the real-space position of the features that were categorized by the semantic segmentation model (Tanaka, [0072]).
Regarding claim 10, Mahnken teaches of a system for generating and displaying a virtual bottom view associated with a vehicle ("systems…for generating and presenting underbody view of a vehicle", [0001]), the system comprising: one or more cameras configured to generate image data associated with a parking zone ("The methods is next operative to capture 330 an image of the FOV using a camera wherein the FOV includes the off-road surface", [0051], implicit that a parking zone falls within an off-road surface); one or more vehicle sensors configured to generate movement data associated with movement of the vehicle ("The processor 240 is further operative to receive vehicle dynamics data, such as steering angle, IMU 233 data, wheel speed, suspension data and inclinometer data to generate a kinematic model of the vehicle", [0046]); and one or more processors ("a processor", [0006]) programmed to: receive the image data from the one or more cameras generated during movement of the vehicle ("The methods is next operative to capture 330 an image of the FOV using a camera wherein the FOV includes the off-road surface", [0051], implicit that a parking zone falls within an off-road surface); execute a semantic segmentation model on the received image data to categorize a portion of the received image data as being associated with an object in the parking zone ("Image processing technique may be used to identify and locate objects within the FOV", [0039]); generate the virtual bottom view based on the image data ("generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046]), and wherein the virtual view includes the three- dimensional view of the object ("generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046], "The system is operative to use various sensors…capable of detecting and mapping various external surfaces", [0039]); and display the virtual bottom view on a vehicle display, enabling a user to see the three-dimensional view of the object when the object is not within the current field of view ("generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046], "…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, Mahnken does not teach of execute a Structure from Motion (SfM) model on the categorized portion of the image data and the movement data, wherein the execution of the SfM model generates a three- dimensional view of the object; wherein the virtual bottom view includes a virtual view of an area of the parking zone that is not within a current field of view of any camera of the vehicle including the one or more cameras; during a parking event; and when the object is not within the current field of view of any camera of the vehicle and is located beneath the vehicle.
Tanaka, in the same field of endeavor, teaches of execute a Structure from Motion (SfM) model on the categorized portion of the image data and the movement data, wherein the execution of the SfM model generates a three- dimensional view of the object ("In the section of the road in which the vehicle 2 is traveling, the position estimation unit 44 of the present embodiment estimates the positions of the vehicle 2 at the times of generation of respective images and the positions of an individual detected feature relative to these positions in accordance with the technique of "structure from motion (SfM)". More specifically, the position estimation unit 44 estimates the positional relationship between a feature of interest and the vehicle 2, based on the position of the feature of interest in each of two or more images obtained at different timings during travel of the vehicle 2 and the distance traveled by the vehicle 2 between the timings of generation of the images", [0061], the generating of a 3D view of an object is implied with the use of SfM); and during a parking event ("a feature is detected from a panoramic image at the time of parking the vehicle", [0068]).
However, Tanaka does not teach of wherein the virtual bottom view includes a virtual view of an area of the parking zone that is not within a current field of view of any camera of the vehicle including the one or more cameras; and when the object is not within the current field of view of any camera of the vehicle and is located beneath the vehicle.
Ren, in the same field of endeavor, teaches of wherein the virtual bottom view includes a virtual view of an area of the parking zone that is not within a current field of view of any camera of the vehicle including the one or more cameras ("virtually reconstruct the area under the vehicle", [0088]); and when the object is not within the current field of view of any camera of the vehicle and is located beneath the vehicle ("existing vehicle Surround View Systems can only visualize front, left, rear, and right sides of a vehicle, leaving a significant blind spot: the area under the vehicle, which may be multiple square meters or more", [0008], "cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area", [0014], implicit that the object is not in the field of view of any camera of the vehicle since the image data is cached to later reconstruct the area under the vehicle).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the method for generating and displaying the bottom view of a vehicle of Mahnken with the teaching of Tanaka to utilize a Structure from Motion (SfM) model and real-time view of the second region and a plurality of cameras during parking and the teaching of Hu to specify the object/area is not in any camera field of view and underneath the vehicle with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the accuracy of real-space position of the objects (Tanaka, [0061]) and allows the system to overcome blind spots during parking and maneuvering (Tanaka, [0078]; Hu, [0236]).
Regarding claim 12, modified Mahnken teaches of all limitations of claim 10 as stated above, additionally, wherein the virtual bottom view includes a transparent image of the vehicle ("This "invisible underbody" may provide the vehicle operator a view of the underbody of the off-road vehicle", [0037]), and is displayed beneath the transparent image ("The exemplary embodiment is operative to generate a three-dimensional terrain map of the off-road surface 120 relative to the critical underbody components of the vehicle…This "invisible underbody" map provide the vehicle operator a view of the underbody of the off-road vehicle", [0036]-[0037]).
However, modified Mahnken does not teach of wherein the area of the parking zone that is not within the current field of view of the one or more cameras.
Tanaka, in the same field of endeavor, teaches of wherein the area of the parking zone that is not within the current field of view of the one or more cameras ("…detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the virtual bottom view of modified Mahnken with the teachings of Tanaka that the parking zone area is not within the current field of view with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to define the utility and benefit of having a virtual bottom view of a vehicle (i.e., solving the problem of blind spots (areas not within the current field of view)).
Regarding claim 13, modified Mahnken teaches of all limitations of claim 10 as stated above, additionally, wherein the virtual bottom view includes a bird-eye- view of the parking zone ("Various image views may include a bird's eye view, vehicle side profile, or driver's view with invisible engine hood", [0036]).
Regarding claim 14, modified Mahnken teaches of all limitations of claim 10 as stated above, additionally, wherein the virtual bottom view includes a perspective view of the parking zone ("Various image views may include a bird's eye view, vehicle side profile, or driver's view with invisible engine hood", [0036]).
Regarding claim 16, Mahnken teaches of a method of generating and displaying a virtual bottom view associated with a vehicle ("methods…for generating and presenting underbody view of a vehicle", [0001]), the method comprising: generating, via one or more cameras and during a first time period, first image data associated with a first region of a parking zone external to the vehicle ("The methods is next operative to capture 330 an image of the FOV using a camera wherein the FOV includes the off-road surface", [0051], implicit that a parking zone falls within an off-road surface), wherein the first region of the parking zone includes an object ("objects within the image, such as rock edges, obstructions, or the like", [0051]); generating, via one or more movement sensors, movement data associated with movement of the vehicle ("The processor 240 is further operative to receive vehicle dynamics data, such as steering angle, IMU 233 data, wheel speed, suspension data and inclinometer data to generate a kinematic model of the vehicle", [0046]) while the first image data is generated ("The processor 240 is then operative to coordinate the kinematic model of the vehicle with the three-dimensional model of the terrain and to generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046]), implicit that movement data is generated while or after the image data has already been generated); executing a synthesis model to synthesize the view with the three-dimensional view ("A composited three-dimensional representation may be generated in response to a combination of the image and the LiDAR depth map", [0064], "Images of the FOV captured by the camera 220 may be correlated with the three-dimensional depth map to generate an augmented image or a virtual three-dimensional representation of the vehicle proximate area", [0041]); and generating, for display on a vehicle display, the virtual bottom view, wherein the virtual bottom view is generated based on the synthesized view and the three-dimensional view, enabling a user to see a three-dimensional virtual view of the object when the object is not in the current field of view ("generate an augmented image representing a three-dimensional representation of the terrain with a generated image of a vehicle driveline", [0046], "…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, Mahnken does not teach of executing a Structure from Motion (SfM) model based on the first image data generated during the first time period and the associated movement data, wherein the executing of the SfM model generates a three-dimensional view associated with the object; generating, via the one or more cameras and during a second time period subsequent the first time period, a real-time view of a second region of the parking zone within a current field of view of the one or more cameras, wherein the object is not in the current field of view of any camera of the vehicle during the second time period and is located beneath the vehicle; during the second time period in which the object is not in the current field of view of any camera of the vehicle; and when the object is not within the current field of view of any camera of the vehicle during a parking event and is located beneath the vehicle.
Tanaka, in the same field of endeavor, teaches of executing a Structure from Motion (SfM) model based on the first image data generated during the first time period and the associated movement data, wherein the executing of the SfM model generates a three-dimensional view associated with the object ("In the section of the road in which the vehicle 2 is traveling, the position estimation unit 44 of the present embodiment estimates the positions of the vehicle 2 at the times of generation of respective images and the positions of an individual detected feature relative to these positions in accordance with the technique of "structure from motion (SfM)". More specifically, the position estimation unit 44 estimates the positional relationship between a feature of interest and the vehicle 2, based on the position of the feature of interest in each of two or more images obtained at different timings during travel of the vehicle 2 and the distance traveled by the vehicle 2 between the timings of generation of the images", [0061], the generating of a 3D view of an object is implied with the use of SfM); generating, via the one or more cameras and during a second time period subsequent the first time period, a real-time view of a second region of the parking zone within a current field of view of the one or more cameras ("detecting one or more predetermined features from a first image representing a first area around the vehicle in a section of the road in which the vehicle is traveling…and detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006], "while" implies real-time); and during a parking event ("a feature is detected from a panoramic image at the time of parking the vehicle", [0068]).
However, Tanaka does not teach of wherein the object is not in the current field of view of any camera of the vehicle during the second time period and is located beneath the vehicle; during the second time period in which the object is not in the current field of view of any camera of the vehicle; and when the object is not within the current field of view of any camera of the vehicle and is located beneath the vehicle.
Hu, in the same field of endeavor, teaches of wherein the object is not in the current field of view of any camera of the vehicle during the second time period and is located beneath the vehicle ("existing vehicle Surround View Systems can only visualize front, left, rear, and right sides of a vehicle, leaving a significant blind spot: the area under the vehicle, which may be multiple square meters or more", [0008], "cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area", [0014], implicit that the object is not in the field of view of any camera of the vehicle since the image data is cached to later reconstruct the area under the vehicle); during the second time period in which the object is not in the current field of view of any camera of the vehicle ("existing vehicle Surround View Systems can only visualize front, left, rear, and right sides of a vehicle, leaving a significant blind spot: the area under the vehicle, which may be multiple square meters or more", [0008], "cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area", [0014], implicit that the object is not in the field of view of any camera of the vehicle since the image data is cached to later reconstruct the area under the vehicle); and when the object is not within the current field of view of any camera of the vehicle and is located beneath the vehicle ("existing vehicle Surround View Systems can only visualize front, left, rear, and right sides of a vehicle, leaving a significant blind spot: the area under the vehicle, which may be multiple square meters or more", [0008], "cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area", [0014], implicit that the object is not in the field of view of any camera of the vehicle since the image data is cached to later reconstruct the area under the vehicle).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the method for generating and displaying the bottom view of a vehicle of Mahnken with the teaching of Tanaka to utilize a Structure from Motion (SfM) model and real-time view of the second region and a plurality of cameras during parking and the teaching of Hu to specify the object not in any camera field of view and underneath the vehicle with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the accuracy of real-space position of the objects (Tanaka, [0061]) and allows the system to overcome blind spots during parking and maneuvering (Tanaka, [0078]; Hu, [0236]).
Regarding claim 17, modified Mahnken teaches of all limitations of claim 16 as stated above, additionally, wherein the execution of the synthesis model further synthesizes the real-time view generated during the second time period with the first image data generated during the first time period ("A composite three-dimensional representation may be generated in response to a combination or the image and the LiDAR depth map", [0064], "The images may be stored in memory 450", [0060], stored in memory implies that image data has occurred in the past, or the "first time period", "the augmented image is generated in response to the host vehicle being located over the off-road surface", [0036]).
Regarding claim 18, modified Mahnken teaches of all limitations of claim 16 as stated above, additionally, further comprising: displaying a transparent image of the vehicle on the virtual bottom view, wherein the first region of the parking zone is displayed beneath the transparent image ("The exemplary embodiment is operative to generate a three-dimensional terrain map of the off-road surface 120 relative to the critical underbody components of the vehicle…This "invisible underbody" map provide the vehicle operator a view of the underbody of the off-road vehicle", [0036]-[0037]).
Regarding claim 19, modified Mahnken teaches of all limitations of claim 18 as stated above, additionally, wherein the three-dimensional view associated with the object is displayed beneath the transparent image ("The three-dimensional terrain model and the vehicle pose estimation may then be used to generate an augmented image wherein the augmented image may include a graphical representation of a vehicle suspension system, vehicle driveline, and wheels overlaid on to an image of the underbody terrain", [0057]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Mahnken, Tanaka, and Hu as applied above, and further in view of Ren (US20230391218A1), hereinafter Ren.
Regarding claim 11, modified Mahnken teaches of all limitations of claim 10 as stated above, additionally, wherein the one or more processors ("a processor", [0006]) is further programmed to: generate a view of the parking zone based on the image data ("The methods is next operative to capture 330 an image of the FOV using a camera wherein the FOV includes the off-road surface", [0051], implicit that a parking zone falls within an off-road surface); synthesize the view of the parking zone and the three-dimensional view of the object to yield a synthesized view ("A composited three-dimensional representation may be generated in response to a combination of the image and the LiDAR depth map", [0064], "Images of the FOV captured by the camera 220 may be correlated with the three-dimensional depth map to generate an augmented image or a virtual three-dimensional representation of the vehicle proximate area", [0041]); and display the synthesized view on the vehicle display ("The processor 240 is then operative to…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, modified Mahnken does not explicitly teach of a real-time view; and display during the parking event.
Tanaka, in the same field of endeavor, teaches of a real-time view ("…detect the one or more predetermined features from a second image representing a second area closer to the vehicle than the first area in a section of road in which the vehicle is parked, the second image being generated by a second camera provided on the vehicle", [0006], "while" implies real-time).
However, Tanaka does not teach of display during the parking event.
Ren, in the same field of endeavor, teaches of display during a parking event (“This visualization may assist and facilitate a variety of driving maneuvers, such as smoothly entering or exiting a parking spot without hitting vulnerable road users like pedestrians or objects such as a road curb or other vehicles”, [0002]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the system for generating and displaying the bottom view of a vehicle of modified Mahnken with the teachings of Tanaka to utilize a real-time view and the teaching of Ren to display during a parking event with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the accuracy of the system by maintaining up to date information that is most representative of the real-space positions of the features within view (Tanaka, [0008]). The modification would further increase the efficiency of the system by effectively providing the driver with visual representation of the surrounding area during parking, where the driver needs information regarding where their line of sight is occluded (Ren, [0002]).
Claims 8-9, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mahnken, Tanaka, and Hu as applied above, and further in view of Sham (US20180001930A1), hereinafter Sham.
Regarding claim 8, modified Mahnken teaches of all limitations of claim 7 as stated above, additionally, wherein the semantic segmentation model categorizes the object ("Image processing techniques may be used to identify and locate objects within the FOV", [0039]).
However, modified Mahnken does not teach of the object as a vehicle wireless charging unit.
Sham, in the same field of endeavor, teaches of the object as a vehicle wireless charging unit ("The wireless charging may be facilitated after the vehicle is automatically parked in accordance with the disclosure. In some examples, a parking spot in accordance with the disclosure may be embedded with a pad on the ground", [0022]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have combined the semantic segmentation model of modified Mahnken with the wireless charging unit of Sham to yield predictable results. One of ordinary skill in the art would have combined these elements in order to detect and categorize the vehicle wireless charging unit which can further be used in determining which parking spots have a wireless charging unit or charging unit alignment (Sham, [0023]).
Regarding claim 9, modified Mahnken teaches of all limitations of claim 8 as stated above, additionally, wherein the processor ("a processor", [0006]) is further programmed to: issue autonomous vehicle control commands to maneuver the vehicle ("The vehicle controller 230 may generate a control signal for coupling to other vehicle system controllers, such as an accelerator controller 255, a brake controller 260 and a steering controller 270 in order to control the operation of the vehicle in response to the ADAS algorithm. The vehicle controller may be operative to adjust the speed of the vehicle by reducing the accelerator via the accelerator controller 255 or to apply the friction brakes vias the brake controller 260 in response to a control signal generated by the processor 240", [0048]); and wherein the virtual bottom view is displayed during the maneuver ("The processor 240 is then operative to…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, modified Mahnken does not teach to maneuver the vehicle such that a portion of the vehicle aligns with the vehicle wireless charging unit.
Sham, in the same field of endeavor, teaches to maneuver the vehicle such that a portion of the vehicle aligns with the vehicle wireless charging unit ("The vehicle 500 may continue backing up and transition to another position…in which the parking maneuver is completed…various sensors are used to confirm its proper alignment", [0086]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the autonomous vehicle maneuvering of modified Mahnken with the teaching of Sham to align the vehicle with the wireless charging unit with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to increase the performance of the charging capacity by aligning the vehicle with the charging unit, which is the optimal position for wireless charging.
Regarding claim 15, modified Mahnken teaches of all limitations of claim 10 as stated above, additionally, wherein: the semantic segmentation model determines that there is an object ("Image processing techniques may be used to identify and locate objects within the FOV", [0039]); the one or more processors is further programmed to issue autonomous vehicle control commands to maneuver the vehicle ("The vehicle controller 230 may generate a control signal for coupling to other vehicle system controllers, such as an accelerator controller 255, a brake controller 260 and a steering controller 270 in order to control the operation of the vehicle in response to the ADAS algorithm. The vehicle controller may be operative to adjust the speed of the vehicle by reducing the accelerator via the accelerator controller 255 or to apply the friction brakes vias the brake controller 260 in response to a control signal generated by the processor 240", [0048]); and the virtual bottom view is displayed during the maneuver ("The processor 240 is then operative to…couple to the augmented image to the user interface 235 or other display for presentation to a vehicle operator", [0046]).
However, modified Mahnken does not teach that the object is a vehicle wireless charging unit; and to maneuver the vehicle such that a portion of the vehicle aligns with the vehicle wireless charging unit.
Sham, in the same field of endeavor, teaches that the object is a vehicle wireless charging unit ("The wireless charging may be facilitated after the vehicle is automatically parked in accordance with the disclosure. In some examples, a parking spot in accordance with the disclosure may be embedded with a pad on the ground", [0022]); and to maneuver the vehicle such that a portion of the vehicle aligns with the vehicle wireless charging unit ("The vehicle 500 may continue backing up and transition to another position…in which the parking maneuver is completed…various sensors are used to confirm its proper alignment", [0086]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have modified the semantic segmentation model, autonomous vehicle maneuvering, and displaying of modified Mahnken with the teachings of Sham of the vehicle wireless charging unit and to align the vehicle with the wireless charging unit with reasonable expectations of success. One of ordinary skill in the art would have been motivated to make this modification in order to detect and categorize the vehicle wireless charging unit which can further be used in determining which parking spots have a wireless charging unit or charging unit alignment (Sham, [0023]), and to increase the performance of the charging capacity by aligning the vehicle with the charging unit, which is the optimal position for wireless charging.
Regarding claim 20, modified Mahnken teaches of all limitations of claim 16 as stated above, further comprising: executing a semantic segmentation model on the first image data to categorize the object ("Image processing technique may be used to identify and locate objects within the FOV", [0039]).
However, modified Mahnken does not teach of the object as a vehicle wireless vehicle charging unit.
Sham, in the same field of endeavor, teaches of the object as a vehicle wireless vehicle charging unit ("The wireless charging may be facilitated after the vehicle is automatically parked in accordance with the disclosure. In some examples, a parking spot in accordance with the disclosure may be embedded with a pad on the ground", [0022]).
Therefore, one of ordinary skill in the art, before the effective filing date of the claimed invention, would have combined the semantic segmentation model of modified Mahnken with the wireless charging unit of Sham to yield predictable results. One of ordinary skill in the art would have combined these elements in order to detect and categorize the vehicle wireless charging unit which can further be used in determining which parking spots have a wireless charging unit or charging unit alignment (Sham, [0023]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABIGAIL LEE ESPINOZA whose telephone number is (571)272-4889. The examiner can normally be reached Monday - Friday 9:00 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached at (571) 270-5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ABIGAIL LEE ESPINOZA
Examiner
Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights