Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This office action is in response to an amendment filed on 2/17/2026. Claims 1-11 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7, and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over JIRALERSPONG et al. (US 2023/0015466 Al) in view of Becker et al. (US 2010/0268458 Al)..
Regarding Claim 1 JIRALERSPONG teaches A method of displaying images on a display screen of a motor vehicle, (Pg. 23 – [0042] – “In addition, a display device or a speaker device is used to transmit information to a driver or the like” &see Also Pg. 21 – [0001] – “The present invention relates to a vehicle control system and a vehicle control method for changing a traveling position of a vehicle according to a visibility rate of an external environment sensor at the time of automatic driving or driving assistance control of a vehicle” ) said method being implemented by a display system comprising at least one first imager suitable for capturing images outside the vehicle, (Pg. 22 – [0038] –“an image sensor such as a camera that images the outside. In the own vehicle Vo of the present example, as the external environment sensor 2, a stereo camera and a millimeter wave radar are provided in the front, laser radars are provided on the left and right” (equates to said method being implemented by a display system comprising at least one first imager suitable for capturing images outside the vehicle as the quote shows the vehicle including camera outside of the vehicle for capturing the environment.)) at least one telemetry device for detecting obstacles, (Pg. 24 – [0051] –“ an object around the own vehicle is detected based on information obtained from the external environment sensor 2.” & See Also Pg. 22 – [0038] –“an image sensor such as a camera that images the outside. In the own vehicle Vo of the present example, as the external environment sensor 2, a stereo camera and a millimeter wave radar are provided in the front, laser radars are provided on the left and right” (equates to at least one telemetry device for detecting obstacles, as the quote shows the detection of objects is done by the environmental sensor which includes a radar sensor and thus a telemetry device. )) a processing unit connected to said at least one first imager and to said at least one telemetry device (Pg. 24 – [0052] –“ Furthermore, although details will be omitted, pre-processing such as filtering or feature extraction processing may be performed on the image acquired from the camera before inputting the image to the discriminator” & See Also Pg. 22 – [0038] – “The external environment sensor 2 is an apparatus that acquires information (also referred to as recognition information) of the outside of the own vehicle Y0 , and includes, for example, one or more of an optical wave sensor that detects an optical wave such as an infrared ray, and electromagnetic wave sensor that detects an electromagnetic wave such as a millimeter wave, and an image sensor such as a camera that images the outside.” & See Also Pg. 24 – [0056] –“ converts an image of the front of the own vehicle acquired from the external environment sensor 2 (stereo camera) illustrated in FIG. 3A into an overhead view image (at this time point, object Ob does not exist) illustrated in FIG. 3B through a known image processing method.” (equates to a processing unit connected to said at least one first imager and to said at least one telemetry device as the first quote shows the camera image being processed and the second quote shows the environmental sensors may include an Infrared camera and the last quote shows how the environmental sensor is put through processing for object detection.)): capturing images via said first imager, (Pg. 24 – [0052] –“ To specify the object, which type on the database the object belongs to is specified, for example by providing images acquired from a stereo camera to an discriminator machine learnt in advance” (equates to capturing images via said first imager as the quote shows the camera images being transmitted for processing and thus an image is taken to do so.)) detecting whether at least one obstacle is present via said telemetry device (Pg. 24 – [0051] –“ an object around the own vehicle is detected based on information obtained from the external environment sensor 2.” & See Also Pg. 22 – [0038] –“an image sensor such as a camera that images the outside. In the own vehicle Vo of the present example, as the external environment sensor 2, a stereo camera and a millimeter wave radar are provided in the front, laser radars are provided on the left and right” (equates to detecting whether at least one obstacle is present via said telemetry device as the quote shows an environmental sensor comprising a radar wherein the result of the sensor is used for object detection.)) and, when said at least one obstacle is present, receiving distance data between said at least one detected obstacle and the vehicle, (Pg. 24 – [0051] –“ In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.”) determining whether the quality of visibility outside the vehicle is below a defined threshold, (Pg. 22 – [0033] – “On the other hand, when the predicted visibility rate is less than or equal to the threshold value.”) determining whether predefined conditions representative of a hazard for the vehicle are met, (Pg. 22 – [0034] – “That is, by making it possible to sense a blind spot area created by the presence of the object, an object (potential risk) hidden behind something can be detected and the vehicle can be appropriately controlled so as to avoid a collision with the object” & See Also Pg. 24 – [0051] –“ In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.” (equates to determining whether predefined conditions representative of a hazard for the vehicle are met as the first quote shows how the vehicle is maneuvered away from the object based on the detection status ensuring a safe travel thus a determination of a hazard is made within the moving of the vehicle away from the obstacle, and wherein the second quote shows a distance away the vehicle is from the obstacle. This is in line with hazard conditions as the specification discloses the hazard sensing based on distance data – Specification - [0015]. )) the hazard originating from said at least one detected obstacle, (Pg. 22 – [0034] – “That is, by making it possible to sense a blind spot area created by the presence of the object, an object (potential risk) hidden behind something can be detected and the vehicle can be appropriately controlled so as to avoid a collision with the object”) said predefined conditions comprising at least the received distance data, (Pg. 24 – [0051] –“ In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.”) and when the predefined conditions representative of a hazard are met and when the visibility quality is below the defined threshold, (Pg. 22 – [0034] – “That is, by making it possible to sense a blind spot area created by the presence of the object, an object (potential risk) hidden behind something can be detected and the vehicle can be appropriately controlled so as to avoid a collision with the object” & See Also Pg. 24 – [0051] –“ In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.” (Equates to when the predefined conditions representative of a hazard are met as the distance of the object is used to determine the vehicle position with the external obstacle) & See Also Pg. 22 – [0033] – “On the other hand, when the predicted visibility rate is less than or equal to the threshold value.” (equates to when the visibility quality is below the defined threshold as a prediction about visibility or a determination is made here.))
Yet JIRALERSPONG fails to teach and a display screen connected to the processing unit, the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, at least one synthesis image to signal said at least one detected obstacle.
Becker teaches and a display screen connected to the processing unit (Pg. 2 - Fig. 1 – (162 & 176) & See Also Pg. 11 – [0032] – “The IPDF 160 comprises a flight computer 162 and a display device 176” & See Also Pg. 11 – [0032] – “The flight computer 162 is used to implement the image rendering engine 164. The image rendering engine 164 is implemented in software 168 that is executed by a suitable processor 172.” (equates to and a display screen connected to the processing unit as the display is provided within the 160 unit and the display is shown to be connected to the 162 computer wherein the computer has a processor.)) the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, (Pg. 11 – [0032] – “image rendering engine 164 overlays the obstacle overlay image onto the terrain image” & See Also Pg. 11 & 12 – [0033] – “The display device 176 displays the composite image of the terrain image and the obstacle image overlay a user (such as a pilot).” & See Also Pg. 12 – [0035] – “The display device 176 displays the composite image to a pilot, and corrections for tilt of the aerial vehicle are made real-time.” & See Also Pg. 1 -Abstract – “Obstacle data pertaining to a set of obstacles ahead of the aerial vehicle is determined with a forward looking sensor. An obstacle overlay image is generated and overlain onto the terrain image to generate a composite image.” (equates to the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, as the quotes show a display of an image comprising a captured image of the terrain and the superimposed version with the object on the terrain. Last quote shows the displaying of the image in real time.)) at least one synthesis image to signal said at least one detected obstacle. (Pg. 12 – [0041] – “FIGS. 3A-3D are images depicting examples of enhanced vision system (EYS) displays produced by the IPFD of FIG. 1. FIG. 3A is a pictorial representation of the obstacles detected by the forward looking radar 135. Among the obstacles detected are a fence 302-A, trucks 304-A, 306-A and 308-A, a helicopter 310-A and a house 312-A. FIG. 3B shows a radar image displayed by an EYS 100 corresponding to the obstacles detected by the forward looking radar 135 in FIG. 3A. In other words, FIG. 3B illustrates the same terrain image from FIG. 3A as well as the obstacle overlay image. Thus, the fence 302-A detected by the forward looking radar 135 in FIG. 3A is displayed as 302-B in FIG. 3B. Likewise the trucks 304-A, 306-A and 308-A appear as obstacles 304-B, 306-B and308-B, respectively. The helicopter 310-A shows up as obstacle 310-B and the house 312-A is displayed as obstacle 312-B” (equates to at least one synthesis image to signal said at least one detected obstacle as the specification discloses the synthesis image to contain “overlay representative of the identified obstacle” as seen by [0070] this quote shows an overlay image of the radar data being displayed of the detected objects wherein the images representing each object is displayed for the driver’s use.)) It would have been an advantageous addition to the system disclosed by JIRALERSPONG to disclosed and a display screen connected to the processing unit, the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, at least one synthesis image to signal said at least one detected obstacle as these limitations allow for an increased user experience of seeing the detected objects around the vehicle and showing on the display in a meaningful way allowing for an easier time navigating and alerting the driver of obstacles that may be in the way of the driving path.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include and a display screen connected to the processing unit, the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, at least one synthesis image to signal said at least one detected obstacle as these limitation allow for a visual feedback way of interpreting the surroundings that the driver may not see immediately without the said sensors installed into the vehicle.
Regarding Claim 7 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, wherein the step of determining whether predefined conditions have been met comprises the following steps: calculating the distance between said obstacle and the vehicle from the distance data (Pg. 24 – [0051] –“ In order to recognize an object, first, an object around the own vehicle is detected based on information obtained from the external environment sensor 2. In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.” (equates to wherein the step of determining whether predefined conditions have been met comprises the following steps: calculating the distance between said obstacle and the vehicle from the distance data as the quote shows a distance of the object being detected away from the vehicle. )), and comparing the calculated distance with a first defined distance value. (Pg. 27 – [0084] – “On the other hand, by applying the third example, even when the detectable range of the sensor is changed based on the weather, the lateral position of the own vehicle Y0 can be offset and a trajectory that is more easily sensed can be generated, so that traveling safety can be improved” & See Also Pg. 27 – [0083] – “In the first example, when the detectable range of the external environment sensor 2 is narrowed by the weather, there is a possibility that even the best visibility rate under the weather condition may not be greater than or equal to a default threshold value ( e.g., 80% ).” (equates to and comparing the calculated distance with a first defined distance value as the quote shows a detectable range of the sensor being impaired by the visibility and thus the range at which the object is detected is affected by the visibility whereby an offset or first defined distance that ensures safety of the trajectory can be implemented to move the host vehicle away from obstacle.))
Regarding Claim 9 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, wherein the step of determining whether predefined conditions have been met comprises the following steps: calculating an amplitude and/or orientation of the speed of said obstacle, (Pg. 28 – [0098] – “a safety area for avoiding collision with the object jumping out is calculated based on the collision danger calculated in step S21, the speed of the own vehicle Y0 , and the maximum speed of the object jumping out.” (equates to calculating an amplitude and/or orientation of the speed of said obstacle as the quote shows how the hazard or collision avoidance is done considering the maximum speed of the object nearby wherein the amplitude of the speed would be the maximum speed that the object is detected traveling at.)) and comparing the calculated amplitude with a defined amplitude and/or comparing the calculated orientation with a defined orientation range. (Pg. 28 – [0099] – “As illustrated in FIG. 15A, the safety area is an area defined by an inter-vehicle distance in a longitudinal direction extending from the own vehicle YO in the advancing direction and an inter-vehicle distance in a lateral direction extending perpendicularly to the advancing direction of the vehicle. In addition, the safety area changes depending on the speed of the own vehicle YO and the maximum speed of the object jumping out.” (equates to and comparing the calculated amplitude with a defined amplitude and/or comparing the calculated orientation with a defined orientation range as the range of the safe zone for vehicle passage is determined based on the speed of the object vehicle and therefor the amplitude of speed is compared to the range of safe travel and the range of safe travel is then set based on the amplitude of the speed of the object.))
Regarding Claim 10 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, wherein the step of determining the quality of visibility outside the vehicle is carried out based on images captured by the first imager and/or distance data received. (Pg. 25 – [0061] – “Subsequently, in step S3, the trajectory planning unit 63 calculates the actual detection range and the visibility rate (=actual detection range/detectable range) of the external environment sensor 2 at the reference target point Pl based on the input information from the external environment sensor 2, the recognition result from the recognizing unit 61, and the driving action information from the driving action planning unit 62.” (equates to wherein the step of determining the quality of visibility outside the vehicle is carried out based on images captured by the first imager and/or distance data received as the quote shows a detectable range and thus a distance data being received by the system for calculation of a visibility rate.))
Regarding Claim 11 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, wherein the at least one telemetry device comprises a radar, lidar or infrared light device. (Pg. 24 – [0051] –“ an object around the own vehicle is detected based on information obtained from the external environment sensor 2.” & See Also Pg. 22 – [0038] –“an image sensor such as a camera that images the outside. In the own vehicle Vo of the present example, as the external environment sensor 2, a stereo camera and a millimeter wave radar are provided in the front, laser radars are provided on the left and right”)
Claims 2 - 4 are rejected under 35 U.S.C. 103 as being unpatentable over JIRALERSPONG-Becker and in view of Azuma (JP7735941B2).
Regarding Claim 2 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, further comprising, when the predefined conditions representative of a hazard are not met or when the visibility quality is equal to or greater than the defined threshold, (Pg. 22 – [0033] – “a visibility rate indicating a ratio of an actual detection range with respect to a detectable range of the sensor is predicted based on the recognition result of the object, and when the predicted visibility rate is greater than or equal to a preset threshold value, the own vehicle YO automatically travels along the planned traveling trajectory” (equates to when the predefined conditions representative of a hazard are not met or when the visibility quality is equal to or greater than the defined threshold as the threshold is shown to be greater than a predefined threshold.))
Yet both fail to teach displaying the images captured by the first imager on said area of the display screen.
Azuma teaches displaying the images captured by the first imager on said area of the display screen (Pg. 5 – [0021] – “The HUD projects a display image formed on a display element based on image data acquired from the HCU 10 onto a projection area defined by a projection member in front of the driver's seat” & See Also Pg. 5 – [0024] - “This HCU 10 corresponds to a vehicle display control device” (equates to displaying the images captured by the first imager on said area of the display screen as the quote shows image being displayed after being attained by the HCU.)) It would have been an advantageous addition to the system disclosed by JIRALERSPONG-Becker to include displaying the images captured by the first imager on said area of the display screen as this allows for an unmerged result to be displayed prior to the superimposed image being shown allowing for clarity of vision to be seen before interpreting the other data.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include displaying the images captured by the first imager on said area of the display screen as this limitation allows for a straight forward image of the surrounding environment to be displayed prior to the generated image being displayed.
Regarding Claim 3 JIRALERSPONG-Becker teaches (JIRALERSPONG teaches the following limitations:) The display method according to claim 1, wherein when the predefined conditions representative of a hazard are not met (Pg. 28 – [0098] – “With the calculated safety area, it can be seen how long the inter-vehicle distance should be maintained in order not to collide with the own vehicle YO even if the object jumps out from the blind spot area.” (equates to and when the predefined conditions representative of a hazard are not met as the conditions via which the hazard conditions are not met are when the distance of the obstacle is outside a range that would interfere with the host vehicle and this quote shows the distance being set as to not collide with the obstacle and thus the hazard conditions are not met)) and when the visibility quality is less than said defined threshold, (Pg. 22 – [0033] – “On the other hand, when the predicted visibility rate is less than or equal to the threshold value, the travel trajectory is re-planned so that the visibility rate becomes greater than or equal to the threshold value, and the own vehicle Y0 automatically travels along the re-planned traveling trajectory.” (equates to when the visibility quality is less than said defined threshold as the quote shows a replanning of the trajectory based on the visibility criterion and thus a hazard being mitigated based on the threshold of visibility being lower than the expected value.))
Yet both fail to teach displaying the images captured by the first imager on said area of the display screen.
Azuma teaches displaying the images captured by the first imager on said area of the display screen (Pg. 5 – [0021] – “The HUD projects a display image formed on a display element based on image data acquired from the HCU 10 onto a projection area defined by a projection member in front of the driver's seat” & See Also Pg. 5 – [0024] - “This HCU 10 corresponds to a vehicle display control device” (equates to the method comprises a step of displaying the images captured by said first imager on said area of the display screen.as the quote shows image being displayed after being attained by the HCU)) It would have been an advantageous addition to the system disclosed by JIRALERSPONG-Becker to include displaying the images captured by the first imager on said area of the display screen as this allows for an unmerged result to be displayed prior to the superimposed image being shown allowing for clarity of vision to be seen before interpreting the other data.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include displaying the images captured by the first imager on said area of the display screen as this limitation allows for a straight forward image of the surrounding environment to be displayed prior to the generated image being displayed.
Regarding Claim 4 JIRALERSPONG-Becker teaches (JIRALERSPONG teaches the following limitations:) The display method according to claim 1, and when the predefined conditions representative of a hazard are not met. (Pg. 28 – [0098] – “With the calculated safety area, it can be seen how long the inter-vehicle distance should be maintained in order not to collide with the own vehicle YO even if the object jumps out from the blind spot area.” (equates to and when the predefined conditions representative of a hazard are not met as the conditions via which the hazard conditions are not met are when the distance of the obstacle is outside a range that would interfere with the host vehicle and this quote shows the distance being set as to not collide with the obstacle and thus the hazard conditions are not met))
Yet Both fail to teach wherein the display system further comprises a second imager installed inside the vehicle and adapted to capture images representing the driver of the vehicle, and wherein the method further comprises the following steps: capturing images via said second imager, determining the position of the vehicle drivers gaze from images captured via the second imager, and at least partially reducing a brightness of said area of the display screen when the determined gaze position is not directed toward said area of the display screen
Azuma teaches wherein the display system further comprises a second imager installed inside the vehicle and adapted to capture images representing the driver of the vehicle (Pg. 4 – [0016] – “The interior camera 12 captures an image of a predetermined range within the cabin of the vehicle. It is preferable that the interior camera 12 captures an image of at least an area including the driver's seat of the vehicle” & See Also Pg. 4 – [0017] –“ The control unit detects the presence of an occupant by using image recognition technology. The control unit may detect the presence of an occupant by recognizing the occupant's face” (equates to wherein the display system further comprises a second imager installed inside the vehicle and adapted to capture images representing the driver of the vehicle as the quote shows an image captured of the cabin of the vehicle wherein a control unit recognizes the occupancy of the driver seat and thus the driver of the vehicle. )) and wherein the method further comprises the following steps: capturing images via said second imager, (Pg. 4 – [0016] – “The interior camera 12 captures an image of a predetermined range within the cabin of the vehicle. It is preferable that the interior camera 12 captures an image of at least an area including the driver's seat of the vehicle”) determining the position of the vehicle drivers gaze from images captured via the second imager, (Pg. 4 – [0017] – “The control unit analyzes the captured image to detect facial features of the occupant. The control unit detects the occupant's facial direction based on the detected facial features of the occupant” & See Also Pg. 5 – [0018] – “The control unit may detect the gaze direction of the occupant. In this case, the control unit detects the pupil and corneal reflex from the captured image by image recognition processing. Then, the gaze direction is detected from the detected facial orientation and the positional relationship between the detected pupil and corneal reflex. The gaze direction may be expressed as a straight line starting from the eye point”(equates to determining the position of the vehicle drivers gaze from images captured via the second imager as the quote shows a face being identified and a turning degree of the head being identified which is in line with the gaze of the driver. And further the gaze is detected by the second quote seen.)) and at least partially reducing a brightness of said area of the display screen when the determined gaze position is not directed toward said area of the display screen (Pg. 7 – [0031] – “On the other hand, in the non-visible region within the display region, the luminance is reduced from the default, as indicated by B.” & See Also Pg. 1 – [0007] – “identification unit (102) that identifies a visible area, which is an area that is estimated to be the area that the occupant of the vehicle is likely to gaze at, based on the results of detecting the occupant of the vehicle” (equates to and at least partially reducing a brightness of said area of the display screen when the determined gaze position is not directed toward said area of the display screen as the first quote shows the brightness of the display screen being reduced based on a non-visible region of the display screen being detected and the second quote showing how the gaze of the driver is used to determine the visibility of the screen to be later used for said display screen reduction.)) It would have been an advantageous addition to the method disclosed by JIRALERSPONG-Becker to include wherein the display system further comprises a second imager installed inside the vehicle and adapted to capture images representing the driver of the vehicle, and wherein the method further comprises the following steps: capturing images via said second imager, determining the position of the vehicle drivers gaze from images captured via the second imager, and at least partially reducing a brightness of said area of the display screen when the determined gaze position is not directed toward said area of the display screen as these limitations allow better feedback with the driver to be had within the system and understand how to modulate the display to minimally distract the driver and ensure a safe trip is had.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include wherein the display system further comprises a second imager installed inside the vehicle and adapted to capture images representing the driver of the vehicle, and wherein the method further comprises the following steps: capturing images via said second imager, determining the position of the vehicle drivers gaze from images captured via the second imager, and at least partially reducing a brightness of said area of the display screen when the determined gaze position is not directed toward said area of the display screen as these limitations allow for reduced power to be had by lowering the brightness when unnecessary and allowing for a bright display that isn’t consistently being used to be dimmed and away from the line of sight of the viewer while driving.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over JIRALERSPONG-Becker and in view of Chan et al. (US 9,849,784 Bl).
Regarding Claim 5 JIRALERSPONG-Becker teaches The display method according to claim 1, as previously mapped above.
Yet JIRALERSPONG-Becker fails to teach further comprising a step of generating said at least one synthesis image comprising at least one signaling feature suitable for signaling the presence of the obstacle, the signaling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle.
Chan teaches further comprising a step of generating said at least one synthesis image comprising at least one signaling feature suitable for signaling the presence of the obstacle, the signaling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle. (Pg. 17 – Fig. 14 – 1440 - “Generating a representation of the object for a scene display depicting the external environment of the vehicle, the representation of the object including a feature having a first color” & See Also Pg. 23 – Col. 5 – lines 31-32 - “FIG. 12 is additional example representations of objects in accordance with aspects of the disclosure.” & See Also Pg. 29 – Col. 18 – lines 65-66 – “example view 1200 of rings 1210 and 1212 which would surround a representation of a pedestrian” (equates to further comprising a step of generating said at least one synthesis image comprising at least one signaling feature suitable for signaling the presence of the obstacle, the signaling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle as the quotes show a representation of objects or pedestrians in this case being represented by a geometric figure of a ring as seen by the fig. 12 and the last quote.)) It would have been an advantageous addition to the system disclosed by JIRALERSPONG-Becker to include further comprising a step of generating said at least one synthesis image comprising at least one signaling feature suitable for signaling the presence of the obstacle, the signaling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle as this limitation allows for a consistent way of easily representing known obstacles in the driving path for the vehicle and thus allowing for an easy representation of obstacles to be relayed to the driver for ease of viewing.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include further comprising a step of generating said at least one synthesis image comprising at least one signaling feature suitable for signaling the presence of the obstacle, the signaling feature being one of an arrow, an icon, a geometric figure, an overlay of said obstacle and an outline of said obstacle as this limitation allows for an easy way of representing known objects within a detectable range of space and thus an easily recognizable key to the driver to discerns between objects in the area quickly understanding each levels of importance.
Regarding Claim 6 JIRALERSPONG-Becker-Chan teaches The display method according to claim 5, as previously mapped above.
Yet JIRALERSPONG-Becker fails to teach which further comprising a step of identifying said obstacle from the distance data, and wherein the generating step comprises generating a signaling feature representative of the identified obstacle.
Chan teaches which further comprising a step of identifying said obstacle from the distance data (Pg. 23 – Col 6 – lines 45-49 – “the category of action for the object (moving in the same lane, merging, turning into the lane, etc.), whether the vehicle needs to take an action to avoid the object (change lanes, slow down, tum the wheels, etc.), distance from the vehicle” & See Also Pg. 17 – Fig. 14 & See Also Pg.30 – Col. 20 – lines 38-41 – “110. In this example, sensor data is received from a sensor configured to detect objects in an external environment of a vehicle at block 1410.” & See Also Pg. 30 – Col. 20 – lines 41 -42 – “An object in the vehicle's environment is identified at block 1420” (equates to which further comprising a step of identifying said obstacle from the distance data as the quote shows an object being detected in step 1410 and the data can be that of distance data as supplied from the first quote wherein the last quote shows an object being identified via the data provided at step 1410.)) and wherein the generating step comprises generating a signaling feature representative of the identified obstacle. (Pg. 30 – Col. 20 – lines 44-46 - “representation of the object is generated for a scene display depicting the external environment of the vehicle at block 1440” & See Also Pg. 29 – Col. 18 – lines 65-66 – “example view 1200 of rings 1210 and 1212 which would surround a representation of a pedestrian” (equates to and wherein the generating step comprises generating a signaling feature representative of the identified obstacle as the quote shows the representation being generated via an obstacle detected and the representation maybe be a signaling feature such as the rings provided in the second quote. )) It would have been an advantageous addition to the method disclosed by JIRALERSPONG-Becker to include which further comprising a step of identifying said obstacle from the distance data, and wherein the generating step comprises generating a signaling feature representative of the identified obstacle as this limitation gives an active alert to a representation of the obstacles that may be encountered by the vehicle and thus allow for an easy visual queue of something to avoid within the trajectory plotted.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include which further comprising a step of identifying said obstacle from the distance data, and wherein the generating step comprises generating a signaling feature representative of the identified obstacle as these limitations allow for a simple representation of the detected object to be overlaid on the screen allowing for the driver to see the detected result in an easily digestible manor.
Claims 8 is rejected under 35 U.S.C. 103 as being unpatentable over JIRALERSPONG-Becker and in view of Srivastav (US 2025/0010885 Al) and in further view of Herman et al. (US 2023/0134302 Al).
Regarding Claim 8 JIRALERSPONG-Becker teaches (JIRALERSPONG discloses the following limitations:) The display method according to claim 1, calculating the distance between said obstacle and the vehicle from the distance data, (Pg. 24 – [0051] –“ In order to recognize an object, first, an object around the own vehicle is detected based on information obtained from the external environment sensor 2. In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.” (equates to wherein the step of determining whether predefined conditions have been met comprises the following steps: calculating the distance between said obstacle and the vehicle from the distance data as the quote shows a distance of the object being detected away from the vehicle. ))
Yet JIRALERSPONG-Becker fail to teach wherein the processing unit is adapted to receive at least one datum representative of the temperature outside the vehicle and at least one datum representative of the humidity outside the vehicle, and wherein the step of determining whether the predefined conditions have been met comprises the following steps: receiving at least one temperature datum and/or at least one humidity datum, comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity, comparing said calculated distance with a second defined distance value.
Srivastav teaches wherein the processing unit is adapted to receive at least one datum representative of the temperature outside the vehicle and at least one datum representative of the humidity outside the vehicle (Pg. 15 – [0055] – “wherein the processing unit is adapted to receive at least one datum representative of the temperature outside the vehicle and at least one datum representative of the humidity outside the vehicle” & See Also Pg. 2 – Fig. 1 – 104 - Environmental (equates to processing unit is adapted to receive at least one datum representative of the temperature outside the vehicle and at least one datum representative of the humidity outside the vehicle as the contextual data is seen to include temperature and humidity data wherein the sensors can gather said data.)) and wherein the step of determining whether the predefined conditions have been met comprises the following steps: receiving at least one temperature datum and/or at least one humidity datum, (Pg. 15 – [0055] – “wherein the processing unit is adapted to receive at least one datum representative of the temperature outside the vehicle and at least one datum representative of the humidity outside the vehicle” & See Also Pg. 2 – Fig. 1 – 104 - Environmental) comparing said calculated distance with a second defined distance value. (Pg. 9 – [0016] – “The different types of particulate clouds may be identifiable based on the impact to the lidar data or based on inputs from other sensors such as temperature and humidity sensors that may be used to identify conditions that may lead to a foggy environment or dusty environment” & See Also Pg. 10 – [0019] – “For example, in foggy conditions, the lidar sensor of an autonomous vehicle may be particularly susceptible to detect non-impeding objects as solid surfaces beyond a threshold distance, where the threshold distance is determined based on the density of the fog.” (equates to comparing said calculated distance with a second defined distance value as the quote shows a fog being determined based on humidity and or temperature related values and the threshold distance changing for the vehicle’s navigation based on the determined fog amount.))
Yet all fail to teach comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity.
Herman teaches comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, (Pg. 13 – [0065] – “while an outside temperature is below a temperature threshold, and to actuate a vehicle actuator 110 based on the determined water area 126.” (equates to comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, as the quote shows an outside temperature that is compared to a threshold temperature for further processing to take place.) and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity, (Pg. 13 – [0065] – “while an outside temperature is below a temperature threshold, and to actuate a vehicle actuator 110 based on the determined water area 126.” (equates to and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity as the quote shows the temperature being below a threshold.)) It would have been an advantageous addition to the system disclosed by JIRALERSPONG-Becker- Srivastav to include comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity as these limitations allow for a rigid setpoint to be used as a barometer for whether or not the external conditions should be considered for reconsidering a distance away the host vehicle travel from the object detected rather than solely machine learning and using fog density data to dynamically adjust the distance needed to be away from the obstacle.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date to include comparing said temperature datum with at least one threshold temperature, and/or comparing said humidity datum with at least one threshold humidity, and when said temperature datum is lower than said threshold temperature and/or when said humidity datum is lower than said threshold humidity as having a simple threshold allows from less processing power to be consumed when determining the distance away the vehicle should travel from the detected object.
Response to Arguments
Response to 35 U.S.C. § 103 rejection of claims __ applicant’s amendments to the claim changes the scope. Applicant’s arguments have been considered but are not persuasive.
Applicant argues on page 2 – “FIG. 4). JIRALERSPONG recognizes obstacles and obtains position/distance/speed/type data via sensors such as radar/cameras (Cf. [0051]).
JIRALERSPONG focuses on trajectory planning to widen actual detection range, not on a hazard determination used to control an HMI display. JIRALERSPONG does not teach "predefined conditions representative of a hazard... comprising at least the received distance data" used to gate a display overlay. Its decision logic affects trajectory, not display behavior ([0033], [0054], [0061]-[0063]).
JIRALERSPONG's HMI description does not disclose any overlay on camera images ([0042]).
BECKER overlays an "obstacle overlay image" onto a "terrain image" to form a "composite image" displayed to the pilot ([0032]-[0035]; FIG. 4, blocks 430-450; [0046]). BECKER shows examples of obstacles overlaid on terrain imagery (FIGS. 3A- 3D; [0041]). BECKER operates in an aviation context using PTAN, DTED, forward- looking radar/FLIR. It does not teach visibility-rate gating or road-vehicle trajectory planning ([0020]-[0024]).
BECKER's overlay is onto a synthetic "terrain image" from DTED, not onto "captured images" (live camera imagery) of a road vehicle; and BECKER's overlay is not conditionally gated by both (i) hazard defined via distance and (ii) low visibility. BECKER presents overlays routinely for aviation situational awareness ([0032]-[0035], [0046]).
Thus, even in combination, the teaching of JIRALERSPONG and BECKER fail to teach or suggest: - the conditional, conjunctive display gating based on both hazard (distance-based condition) and visibility below threshold; and - the real-time overlay localized "around" the obstacle within "captured images" from an in-vehicle camera, rather than onto a synthetic terrain map.” As to point A the examiner respectfully disagrees. Applicant asserts that Jilarspong in view of Becker does not teach “when the predefined conditions representative of a hazard are met and when the visibility quality is below the defined threshold,”. During Patent Examination, pending claims must be given their broadest reasonable interpretation consistent with the specification (see MPEP 2111). The broadest reasonable interpretation of the aforementioned amendment is defining a situation when any hazard type of any level is detected and a visibility quality being below a threshold. Jilarspong teaches an object being within a zone in which a collision would happen with the vehicle and a visibility rate being below a threshold (as mapped above in claim 1). Therefor the Examiner respectfully disagrees with the applicants arguments and assert that Jilarspong teaches “when the predefined conditions representative of a hazard are met and when the visibility quality is below the defined threshold,”. Jiralspong teaches: and when the predefined conditions representative of a hazard are met and when the visibility quality is below the defined threshold, (Pg. 22 – [0034] – “That is, by making it possible to sense a blind spot area created by the presence of the object, an object (potential risk) hidden behind something can be detected and the vehicle can be appropriately controlled so as to avoid a collision with the object” & See Also Pg. 24 – [0051] –“ In order to detect an object, for example, a millimeter wave radar is irradiated, and a radio wave reflected back by the object is detected, whereby the distance and direction to the object are detected. In another method, an object is detected based on whether or not a shape of a gauge mark detected by a distance sensor matches a shape of a template stored in advance.” (Equates to when the predefined conditions representative of a hazard are met as the distance of the object is used to determine the vehicle position with the external obstacle) & See Also Pg. 22 – [0033] – “On the other hand, when the predicted visibility rate is less than or equal to the threshold value.” (equates to when the visibility quality is below the defined threshold as a prediction about visibility or a determination is made here.))
Applicant argues on Page 3, “Furthermore, JIRALERSPONG solves poor sensor visibility by trajectory re- planning to widen actual detection range ([0054], [0033]). BECKER solves aviation navigation/awareness by compositing terrain and obstacle overlays ([0032]-[0035], [0046]). The claimed invention solves a different problem which is minimizing nuisance/alarm clutter by only superimposing an obstacle signal on camera images when two conditions are jointly satisfied (hazard + low visibility).
Neither JIRALERSPONG nor BECKER teaches to control overlays based on visibility rate and obstacle distance danger. Indeed, BECKER overlay is not controlled by visibility rate thresholds or danger determinations. JIRALERSPONG control concerns trajectory selection, not display overlays.
Neither JIRALERSPONG nor BECKER teaches to gate overlays by visibility-rate and obstacle-distance hazard. BECKER's overlay is not gated by visibility-rate thresholds or hazard determinations. JIRALERSPONG's gating pertains to trajectory selection, not display overlays.
BECKER's terrain-image overlay pipeline (DTED/PTAN/IMU geo-registration) differs fundamentally from an automotive camera-image overlay localized "around" the obstacle and conditioned by road-vehicle visibility/hazard logic. The combination lacks a reasoned path to adapt BECKER's synthetic terrain overlay to the claimed camera-based overlay behavior in the automotive HMI context.
Advantageously, the joint control system and the localized overlay of the method according to the present claims allow: targeted signaling only when a real danger (based on distance) exists and visibility is reduced, thus decreasing false alerts and screen clutter; improved environmental awareness for the driver thanks to the real-time presentation of images from the camera of the surroundings with localized obstacle signals.” As to point A the examiner respectfully disagrees. Applicant asserts that Becker does not teach “displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, at least one synthesis image to signal said at least one detected obstacle”. During Patent Examination, pending claims must be given their broadest reasonable interpretation consistent with the specification (see MPEP 2111). The broadest reasonable interpretation of the aforementioned limitations is to generate any image type of an area in question, and overlay another image in which an obstacle is detected onto the first image of the area. Becker teaches an image of the forward area of a vehicle being generated and another image in which an obstacle is detected in front of the vehicle and then overlaying the object picture with the terrain image (as mapped above in claim1). Therefor the Examiner respectfully disagrees with the applicants arguments and assert that __reference previously used__ teaches “displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, at least one synthesis image to signal said at least one detected obstacle”. Becker teaches : the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, (Pg. 13 – [0044] – “Once the position of the aerial vehicle is determined a terrain image correlated to the position of the aerial vehicle is generated (block 420). The terrain image is generated by a terrain rendering engine 118. The terrain rendering engine 118 takes the position data, either from GPS 130 or the PIAN radar altimeter 110, along with attitude information and renders an image of the terrain. The terrain image can be provided by a digital map data stored in a memory onboard the aerial vehicle” 7 See Also Pg. 11 – [0032] – “image rendering engine 164 overlays the obstacle overlay image onto the terrain image” & See Also Pg. 11 & 12 – [0033] – “The display device 176 displays the composite image of the terrain image and the obstacle image overlay a user (such as a pilot).” & See Also Pg. 12 – [0035] – “The display device 176 displays the composite image to a pilot, and corrections for tilt of the aerial vehicle are made real-time.” & See Also Pg. 1 -Abstract – “Obstacle data pertaining to a set of obstacles ahead of the aerial vehicle is determined with a forward looking sensor. An obstacle overlay image is generated and overlain onto the terrain image to generate a composite image.” (equates to the display method comprising the following steps: displaying the captured images on an area of the display screen and superimposing, on these captured images and around said at least one detected obstacle, in real time, as the quotes show a display of an image comprising a captured image of the terrain and the superimposed version with the object on the terrain. Last quote shows the displaying of the image in real time.)) at least one synthesis image to signal said at least one detected obstacle. (Pg. 12 – [0041] – “FIGS. 3A-3D are images depicting examples of enhanced vision system (EYS) displays produced by the IPFD of FIG. 1. FIG. 3A is a pictorial representation of the obstacles detected by the forward looking radar 135. Among the obstacles detected are a fence 302-A, trucks 304-A, 306-A and 308-A, a helicopter 310-A and a house 312-A. FIG. 3B shows a radar image displayed by an EYS 100 corresponding to the obstacles detected by the forward looking radar 135 in FIG. 3A. In other words, FIG. 3B illustrates the same terrain image from FIG. 3A as well as the obstacle overlay image. Thus, the fence 302-A detected by the forward looking radar 135 in FIG. 3A is displayed as 302-B in FIG. 3B. Likewise the trucks 304-A, 306-A and 308-A appear as obstacles 304-B, 306-B and308-B, respectively. The helicopter 310-A shows up as obstacle 310-B and the house 312-A is displayed as obstacle 312-B” (equates to at least one synthesis image to signal said at least one detected obstacle as the specification discloses the synthesis image to contain “overlay representative of the identified obstacle” as seen by [0070] this quote shows an overlay image of the radar data being displayed of the detected objects wherein the images representing each object is displayed for the driver’s use.))
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US12384299 . - The system also performs object detection to inform a driver of possible objects obstructing a path of the vehicle and/or feed into a recommendation algorithm for recommending a path trajectory via the visualization. Additionally, the visualization may be enhanced with overlay information (e.g., user interface elements representing vehicle trajectory guidelines, detected objects, etc.).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REECE ANTHONY WAKELY whose telephone number is (571)272-3783. The examiner can normally be reached Monday - Friday 8:30am-6:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hitesh Patel can be reached at (571) 270-5442. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.A.W./Examiner, Art Unit 3667
/Hitesh Patel/Supervisory Patent Examiner, Art Unit 3667
5/4/26
Prosecution Insights