Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The Office Action is in response to the application filed 08/31/2026. Claims 1, 3-11, and 13-20 are presently pending and are presented for examination.
Response to Arguments
Applicant's arguments, see pages 11-14, filed 08/31/2026, regarding the rejection of the claims under 35 U.S.C. 103 in view of Ahn et al. US 20220281317 A1 (“Ahn”) in combination with Singhal US 20160229341 A1 (“Singhal”) have been fully considered but they are not persuasive. Applicant’s arguments fundamentally come down to the interpretation of the term “head movement”, and the applicant claims that the prior art does not teach this element, only teaching that Ahn tracks the eye movement of a user. However, under the broadest reasonable interpretation of the claims as written, a user’s gaze is a type of head movement, since a user’s eye is presumably part of their head. The claims as written do not provide any details that would defy this interpretation. For this reason, the claims are rejected under 35 U.S.C. 103 in view of Ahn in combination with Singhal.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 9, 11-14, 16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ahn et al. US 20220281317 A1 (“Ahn”) in combination with Singhal US 20160229341 A1 (“Singhal”).
Regarding Claim 1. Ahn teaches a system implemented in a vehicle, the system comprising:
at least one rear-view camera (A vehicle with a rear-view mirror image captured by a rear camera [paragraph 57]);
a heads-up display (The heads-up display (HUD) of FIG. 1 [paragraphs 62-63]); and
at least one processor configured to:
capture at least one rear-view image of a region of a surrounding environment that is behind the vehicle, by utilising the at least one rear-view camera (Referring to FIG. 1, the electronic device 1000 may include the transparent screen 1100, a projector 1200, and an eye tracking sensor 1310. However, components included in the electronic device 1000 are not limited to those illustrated in FIG. 1 [paragraph 54]);
generate an image to be displayed via the heads-up display, wherein when generating the image, the at least one processor is configured to generate an image segment of the image by utilising the at least one rear-view image of said region of the surrounding environment (FIG. 6 describes an operating method of the device at 1000 of FIG. 1 [paragraph 107]. For example, the electronic device 1000 may control the projector to project an augmented reality image representing a route guidance signal or navigation information related to driving of the vehicle onto a first region of the transparent screen, a rear-view mirror image captured by a rear camera of the vehicle onto a second region of the transparent screen, and a UI representing information such as weather onto a third region of the transparent screen [paragraph 108]); and
display the image via the heads-up display for producing a synthetic light field augmenting a real-world light field incoming via a windshield of the vehicle (This is the display shown in FIG. 1),
wherein the system further comprises tracking means, wherein the at least one processor is further configured to:
during the given time period, detect, by utilising the tracking means, when at least one user has made a head movement, whilst gazing at a given portion of the heads-up display whereat said image segment of the image is being displayed (FIGS. 7A and &B show a detailed structure and operation of an eye tracking sensor [paragraph 97]. Referring to FIG. 7A, the eye tracking sensor 1310 may track the gaze of the driver based on the positions of rays of reflected light 711, 712, 713, 714, and 715 reflected from an eye of the driver, and thus obtain a gaze vector [paragraph 112]); and
when it is detected that the at least one user has made a head movement, whilst gazing at the given portion of the heads-up display, perform at least one of:
generate a current image segment of a current image to be displayed via the heads-up display, based on a direction and a magnitude of the head movement (FIG. 9 is a flowchart illustrating a method, performed by the electronic device 1000, of performing image warping based on information about a gaze vector, according to an embodiment of the disclosure [paragraph 131]. Operations S910 and S920 shown in FIG. 9 are subdivided operations of operation S620 shown in FIG. 6. Operations S930 and S940 shown in FIG. 9 are subdivided operations of operation S630 shown in FIG. 6. Operation S910 shown in FIG. 9 is performed after operation S610 of FIG. 6 is performed [paragraph 132]);
increase a size of the current image segment, based on the direction and the magnitude of the head movement (In operation S930, the electronic device 1000 may calculate a degree of distortion of an image projected onto the transparent screen, as perceived by a driver, using the information about the magnitudes, angles, and heights of the gaze vectors. In an embodiment of the disclosure, the electronic device 1000 may obtain position coordinates of each of a plurality of pixels included in the image projected onto the transparent screen at which the driver is looking, using the information about the magnitudes, angles, and heights of the first and second gaze vectors. The electronic device 1000 may calculate the degree of distortion of the image that will be perceived by the driver, according to the gaze direction of the driver, by comparing the obtained position coordinates of the plurality of pixels with the position coordinates of the plurality of pixels included in the original image [paragraph 135]).
Ahn does not teach:
repeat the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment;
wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region;
wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region.
However, Singhal teaches, as best can be understood:
repeat the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment (In embodiments of the above configuration, the imaging unit includes a rear camera that captures an image of the area behind the vehicle; a front camera that captures an image of the area in front of the vehicle; a left camera that captures an image of an area on a left side of the vehicle; and a right camera that captures an image of an area on a right side of the vehicle, and the image processing unit generates the first view image on the basis of the images captured by the rear camera, the left camera, and the right camera and generates the second view image on the basis of the images captured by the front camera, the left camera, and the right camera [paragraph 12]);
wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region (paragraph 12);
wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region (FIG. 1A shows the various cameras around the vehicle, which provide the field of view for the display screen. The right and left camera systems at 14 and 16 have wider fields of view than the rear camera at 15. Additionally, the “left side” region could include the sections LC 1, LC2, and LC 3 of FIG. 3B. It would have been obvious to combine the different parts of the left side view into one as a mere joining of parts).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with repeat the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment; wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region; wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region as taught by Singhal so as to allow the vehicle to display traffic along the side of the vehicle, as well as allowing the system to show different fields of view to provide the driver with a full understanding of their surroundings.
Regarding Claim 3. Ahn in combination with Singhal teaches the system of claim 1.
Ahn does not teach:
further comprising at least one side-view camera, wherein the at least one processor is configured to:
capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilizing the at least one side-view camera; and
when generating the image, generate at least one other image segment of the image by utilizing the at least one side-view image of the another region of the surrounding environment.
However, Singhal teaches:
further comprising at least one side-view camera, wherein the at least one processor is configured to:
capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilizing the at least one side-view camera (In embodiments of the above configuration, the imaging unit includes a rear camera that captures an image of the area behind the vehicle; a front camera that captures an image of the area in front of the vehicle; a left camera that captures an image of an area on a left side of the vehicle; and a right camera that captures an image of an area on a right side of the vehicle, and the image processing unit generates the first view image on the basis of the images captured by the rear camera, the left camera, and the right camera and generates the second view image on the basis of the images captured by the front camera, the left camera, and the right camera [paragraph 12]); and
when generating the image, generate at least one other image segment of the image by utilizing the at least one side-view image of the another region of the surrounding environment (FIG. 3B).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising at least one side-view camera, wherein the at least one processor is configured to: capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilizing the at least one side-view camera; and when generating the image, generate at least one other image segment of the image by utilizing the at least one side-view image of the another region of the surrounding environment as taught by Singhal so as to allow the vehicle to display traffic along the side of the vehicle.
Regarding Claim 4. Ahn in combination with Singhal teaches the system of claim 1.
Ahn also teaches:
further comprising an active optical device arranged on an optical path of the real-world light field, wherein the at least one processor is configured to:
determine a given portion of the active optical device that corresponds to a given portion of the heads-up display whereat said image segment of the image is being displayed; and
activate the given portion of the active optical device to decrease a transmission of a part of the real-world light field passing through the given portion of the active optical device (In an embodiment of the disclosure, the processor 1400 may perform image warping by using a transformation function that includes a vertical or horizontal shift, enlargement, reduction, tilting, rotation, or the like, based on the detected gaze direction of the driver [paragraph 61]. This would include, based on the disclosure of paragraph 5, lowering the power projection of the image on the windshield).
Regarding Claim 6. Ahn in combination with Singhal teaches the system of claim 1.
Ahn does not teach:
wherein the at least one processor is configured to adjust at least one of:
a location, a size of said image segment of the image, based on at least one of:
an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed.
However, Singhal teaches:
wherein the at least one processor is configured to adjust at least one of:
a location, a size of said image segment of the image, based on at least one of:
an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed (As illustrated with the help of FIG. 5B, the Camera system 70 has system logic 50 that enables the camera system 70, with functions of: a function 52 to receive ambient light input & adjust display screen intensity, as well as from rain/fog/moisture input [paragraph 84]. FIG. 7B describes adjusting the composite image accounting for different ambient light conditions surrounding the vehicle at step 120, which includes adjusting the visual icon size at step 122).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with wherein the at least one processor is configured to adjust at least one of: a location, a size of said image segment of the image, based on at least one of: an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed as taught by Singhal so as to adjust the display on the windshield in response to ambient lighting conditions.
Regarding Claim 9. Ahn in combination with Singhal teaches the system of claim 1.
Ahn also teaches:
further comprising tracking means and at least one rear-view depth camera, wherein the at least one processor is configured to:
determine a relative location of a first eye and a second eye of at least one user with respect to an image plane of the heads-up display, by utilising the tracking means (The processor 1400 (see FIG. 5) included in the electronic device 1000 may obtain gaze direction information of the left and right eyes of the driver from the eye tracking sensor 1310 [paragraph 61]);
generate a depth image of said region of the surrounding environment, by utilising the at least one rear-view depth camera (Paragraph 145 describes how the projector may form a depth image based in part on the gaze point of the driver. This would only be possible if the rear camera is also a depth camera, as there has to be some means for acquiring depth information for the projector);
reproject the at least one rear-view image from a perspective of the at least one rear-view camera to a perspective of the first eye and of the second eye of the at least one user, based on the relative location of the first eye and the second eye of the at least one user with respect to the image plane of the heads-up display, respectively, wherein the at least one rear-view image is reprojected by utilising the depth image of said region of the surrounding environment (The transparent screen 1100 may include a plurality of inclination angle patterns that direct the path of projected light towards a driver's seat, by reflecting and refracting light of the image projected by the projector 1200. Because of the plurality of inclination angle patterns, the light of the image projected by the projector 1200 may be reflected and refracted in a direction toward a driver seated in the driver's seat [paragraph 56]. The degree of distortion of the image, for example, when viewed by the driver, may depend on at least one of the distance between both eyes of the driver and the transparent screen 1100, the angle of the gaze direction of the driver gazing at the second region 1100-2 of the transparent screen 1100, or the height of the eyes of the driver. In an embodiment of the disclosure, the processor 1400 may obtain, from the gaze direction information obtained by the eye tracking sensor 1310, information about at least one of the distance between both eyes of the driver and the transparent screen 1100, the angle of the gaze direction of the driver gazing at the transparent screen 1100, or the gaze height of the driver, and perform image warping for preventing an image distortion based on the obtained information [paragraph 61]); and
when generating the image segment, generate a first set of pixels and a second set of pixels of the image segment by utilising the at least one rear-view image after reprojecting to the perspective of the first eye and of the second eye, respectively (paragraph 52, where it explains the meaning of image warping in paragraphs 56 and 61),
wherein when the image is displayed via the heads-up display, light produced by the first set of pixels is directed towards the first eye, whilst light produced by the second set of pixels is directed towards the second eye (Shown in FIG. 11, the location of the projection at G(x,y,z) is shown to be at the location where the driver’s eyes converge).
Regarding Claim 11. Ahn teaches a method implemented in a vehicle, the method comprising:
capturing at least one rear-view image of a region of a surrounding environment that is behind the vehicle, by utilising at least one rear-view camera (A vehicle with a rear-view mirror image captured by a rear camera [paragraph 57]);
generating an image to be displayed via the heads-up display, wherein the step of generating the image comprises generating an image segment of the image by utilising the at least one rear-view image of said region of the surrounding environment (FIG. 6 describes an operating method of the device at 1000 of FIG. 1 [paragraph 107]. For example, the electronic device 1000 may control the projector to project an augmented reality image representing a route guidance signal or navigation information related to driving of the vehicle onto a first region of the transparent screen, a rear-view mirror image captured by a rear camera of the vehicle onto a second region of the transparent screen, and a UI representing information such as weather onto a third region of the transparent screen [paragraph 108]); and
displaying the image via the heads-up display for producing a synthetic light field augmenting a real-world light field incoming via a windshield of the vehicle (This is the display shown in FIG. 1);
during the given time period, detecting, by utilising tracking means, when at least one user has made a head movement, whilst gazing at a given portion of the heads-up display whereat said image segment of the image is being displayed (FIGS. 7A and &B show a detailed structure and operation of an eye tracking sensor [paragraph 97]. Referring to FIG. 7A, the eye tracking sensor 1310 may track the gaze of the driver based on the positions of rays of reflected light 711, 712, 713, 714, and 715 reflected from an eye of the driver, and thus obtain a gaze vector [paragraph 112]); and
when it is detected that the at least one user has made a head movement, whilst gazing at the given portion of the heads-up display, performing at least one of:
generating a current image segment of a current image to be displayed via the heads-up display, based on a direction and a magnitude of the head movement (FIG. 9 is a flowchart illustrating a method, performed by the electronic device 1000, of performing image warping based on information about a gaze vector, according to an embodiment of the disclosure [paragraph 131]. Operations S910 and S920 shown in FIG. 9 are subdivided operations of operation S620 shown in FIG. 6. Operations S930 and S940 shown in FIG. 9 are subdivided operations of operation S630 shown in FIG. 6. Operation S910 shown in FIG. 9 is performed after operation S610 of FIG. 6 is performed [paragraph 132]);
increasing a size of the current image segment, based on the direction and the magnitude of the head movement (In operation S930, the electronic device 1000 may calculate a degree of distortion of an image projected onto the transparent screen, as perceived by a driver, using the information about the magnitudes, angles, and heights of the gaze vectors. In an embodiment of the disclosure, the electronic device 1000 may obtain position coordinates of each of a plurality of pixels included in the image projected onto the transparent screen at which the driver is looking, using the information about the magnitudes, angles, and heights of the first and second gaze vectors. The electronic device 1000 may calculate the degree of distortion of the image that will be perceived by the driver, according to the gaze direction of the driver, by comparing the obtained position coordinates of the plurality of pixels with the position coordinates of the plurality of pixels included in the original image [paragraph 135]).
Ahn does not teach:
repeating the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment;
wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region;
wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region.
However, Singhal teaches, as best can be understood:
repeating the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment (In embodiments of the above configuration, the imaging unit includes a rear camera that captures an image of the area behind the vehicle; a front camera that captures an image of the area in front of the vehicle; a left camera that captures an image of an area on a left side of the vehicle; and a right camera that captures an image of an area on a right side of the vehicle, and the image processing unit generates the first view image on the basis of the images captured by the rear camera, the left camera, and the right camera and generates the second view image on the basis of the images captured by the front camera, the left camera, and the right camera [paragraph 12]);
wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region (paragraph 12);
wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region (FIG. 1A shows the various cameras around the vehicle, which provide the field of view for the display screen. The right and left camera systems at 14 and 16 have wider fields of view than the rear camera at 15. Additionally, the “left side” region could include the sections LC 1, LC2, and LC 3 of FIG. 3B. It would have been obvious to combine the different parts of the left side view into one as a mere joining of parts).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with repeating the steps of capturing, generating and displaying for a given time period, wherein said image segment represents a given sub-region of said region of the surrounding environment; wherein the current image segment represents another sub-region of said region of the surrounding environment that overlaps with at least a part of the given sub-region of said region; wherein the current image segment represents yet another sub-region of said region of the surrounding environment whose field of view is larger than a field of view of the given sub-region of said region as taught by Singhal so as to allow the vehicle to display traffic along the side of the vehicle, as well as allowing the system to show different fields of view to provide the driver with a full understanding of their surroundings.
Regarding Claim 13. Ahn in combination with Singhal teaches the method of claim 11.
Ahn does not teach:
further comprising:
capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle by utilising at least one side-view camera; and
when generating the image, generating at least one other image segment of the image by utilising the at least one side-view image of the another region of the surrounding environment.
However, Singhal teaches:
further comprising:
capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle by utilising at least one side-view camera (In embodiments of the above configuration, the imaging unit includes a rear camera that captures an image of the area behind the vehicle; a front camera that captures an image of the area in front of the vehicle; a left camera that captures an image of an area on a left side of the vehicle; and a right camera that captures an image of an area on a right side of the vehicle, and the image processing unit generates the first view image on the basis of the images captured by the rear camera, the left camera, and the right camera and generates the second view image on the basis of the images captured by the front camera, the left camera, and the right camera [paragraph 12]); and
when generating the image, generating at least one other image segment of the image by utilising the at least one side-view image of the another region of the surrounding environment (FIG. 3B).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising: capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle by utilising at least one side-view camera; and when generating the image, generating at least one other image segment of the image by utilising the at least one side-view image of the another region of the surrounding environment as taught by Singhal so as to allow the vehicle to display traffic along the side of the vehicle.
Regarding Claim 14. Ahn in combination with Singhal teaches the method of claim 11.
Ahn also teaches:
further comprising:
determining a given portion of an active optical device that corresponds to a given portion of the heads-up display whereat said image segment of the image is being displayed, wherein the active optical device is arranged on an optical path of the real-world light field; and
activating the given portion of the active optical device to decrease a transmission of a part of the real-world light field passing through the given portion of the active optical device (In an embodiment of the disclosure, the processor 1400 may perform image warping by using a transformation function that includes a vertical or horizontal shift, enlargement, reduction, tilting, rotation, or the like, based on the detected gaze direction of the driver [paragraph 61]. This would include, based on the disclosure of paragraph 5, lowering the power projection of the image on the windshield).
Regarding Claim 16. Ahn in combination with Singhal teaches the method of claim 11.
Ahn does not teach:
further comprising adjusting at least one of:
a location, a size of said image segment of the image, based on at least one of:
an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed.
However, Singhal teaches:
further comprising adjusting at least one of:
a location, a size of said image segment of the image, based on at least one of:
an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed (As illustrated with the help of FIG. 5B, the Camera system 70 has system logic 50 that enables the camera system 70, with functions of: a function 52 to receive ambient light input & adjust display screen intensity, as well as from rain/fog/moisture input [paragraph 84]. FIG. 7B describes adjusting the composite image accounting for different ambient light conditions surrounding the vehicle at step 120, which includes adjusting the visual icon size at step 122).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising adjusting at least one of: a location, a size of said image segment of the image, based on at least one of: an ambient light intensity in the surrounding environment, a weather condition in the surrounding environment, a road condition in the surrounding environment, a traffic condition in the surrounding environment, whether the vehicle is being reversed as taught by Singhal so as to adjust the display on the windshield in response to ambient lighting conditions.
Regarding Claim 19. Ahn in combination with Singhal teaches the method of claim 11.
Ahn also teaches:
further comprising:
determining a relative location of a first eye and a second eye of at least one user with respect to an image plane of the heads-up display, by utilising tracking means (The processor 1400 (see FIG. 5) included in the electronic device 1000 may obtain gaze direction information of the left and right eyes of the driver from the eye tracking sensor 1310 [paragraph 61]);
generating a depth image of said region of the surrounding environment, by utilising at least one rear-view depth camera (Paragraph 145 describes how the projector may form a depth image based in part on the gaze point of the driver. This would only be possible if the rear camera is also a depth camera, as there has to be some means for acquiring depth information for the projector);
reprojecting the at least one rear-view image from a perspective of the at least one rear- view camera to a perspective of the first eye and of the second eye of the at least one user, based on the relative location of the first eye and the second eye of the at least one user with respect to the image plane of the heads-up display, respectively, wherein the at least one rear- view image is reprojected by utilising the depth image of said region of the surrounding environment (The transparent screen 1100 may include a plurality of inclination angle patterns that direct the path of projected light towards a driver's seat, by reflecting and refracting light of the image projected by the projector 1200. Because of the plurality of inclination angle patterns, the light of the image projected by the projector 1200 may be reflected and refracted in a direction toward a driver seated in the driver's seat [paragraph 56]. The degree of distortion of the image, for example, when viewed by the driver, may depend on at least one of the distance between both eyes of the driver and the transparent screen 1100, the angle of the gaze direction of the driver gazing at the second region 1100-2 of the transparent screen 1100, or the height of the eyes of the driver. In an embodiment of the disclosure, the processor 1400 may obtain, from the gaze direction information obtained by the eye tracking sensor 1310, information about at least one of the distance between both eyes of the driver and the transparent screen 1100, the angle of the gaze direction of the driver gazing at the transparent screen 1100, or the gaze height of the driver, and perform image warping for preventing an image distortion based on the obtained information [paragraph 61]); and
when generating the image segment, generating a first set of pixels and a second set of pixels of the image segment by utilising the at least one rear-view image after reprojecting to the perspective of the first eye and of the second eye, respectively (paragraph 52, where it explains the meaning of image warping in paragraphs 56 and 61),
wherein when the image is displayed via the heads-up display, light produced by the first set of pixels is directed towards the first eye, whilst light produced by the second set of pixels is directed towards the second eye (Shown in FIG. 11, the location of the projection at G(x,y,z) is shown to be at the location where the driver’s eyes converge).
Claim(s) 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ahn et al. US 20220281317 A1 (“Ahn”) in combination with Singhal US 20160229341 A1 (“Singhal”) as applied to claims 1 and 11 above, and further in view of Wyona Lars et al. DE 102012019508 A1 (“Wyona Lars”).
Regarding Claim 5. Ahn in combination with Singhal teaches the system of claim 1.
Ahn does not teach:
further comprising tracking means, wherein the at least one processor is configured to:
determine gaze directions of eyes of at least one user, by utilising the tracking means;
detect, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and
when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, perform any one of:
generate the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or
generate an entirety of the image by utilising at least one virtual image.
However, Wyona Lars teaches:
further comprising tracking means, wherein the at least one processor is configured to:
determine gaze directions of eyes of at least one user, by utilising the tracking means;
detect, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and
when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, perform any one of:
generate the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or
generate an entirety of the image by utilising at least one virtual image (Paragraph 41 and 42, FIGS. 4 and 5. “According to a further development, it can be provided that the control unit 22 not only captures the driver's current line of sight, but tracks its development over a period of a few seconds” [paragraph 43]. In this case, the example provided is one in which the driver is observing an animal outside the vehicle. Therefore, if the driver's gaze at the end of a continuous sideways movement on the display screen 17 then there is a high probability that the driver is not at the display on the display screen 17 is interested, but in an object of the environment, due to the movement of the vehicle behind the display screen 17 wandered. In such a case, it may be appropriate not to complicate the tracking of the object by switching to a highly eye-catching presentation, but to leave the representation unchanged. A switch to the non-transparent state takes place according to this development only when an uneven, with the assumption that the driver pursues an object outside the vehicle with his eyes, incompatible eye movement the driver's gaze on the display screen 17 has led. Paragraph 41 in particular describes making the display particularly transparent as a result of the driver looking away).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising tracking means, wherein the at least one processor is configured to: determine gaze directions of eyes of at least one user, by utilising the tracking means; detect, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, perform any one of: generate the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or generate an entirety of the image by utilising at least one virtual image as taught by Wyona Lars so as to allow the system to reduce the distraction of the heads-up display when the driver needs to focus on something else.
Regarding Claim 15. Ahn teaches the method of claim 11.
Ahn does not teach:
further comprising:
determining gaze directions of eyes of at least one user, by utilising tracking means;
detecting, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and
when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, performing any one of:
generating the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or
generating an entirety of the image by utilising at least one virtual image.
However, Wyona Lars teaches:
further comprising:
determining gaze directions of eyes of at least one user, by utilising tracking means;
detecting, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and
when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, performing any one of:
generating the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or
generating an entirety of the image by utilising at least one virtual image (Paragraph 41 and 42, FIGS. 4 and 5. “According to a further development, it can be provided that the control unit 22 not only captures the driver's current line of sight, but tracks its development over a period of a few seconds” [paragraph 43]. In this case, the example provided is one in which the driver is observing an animal outside the vehicle. Therefore, if the driver's gaze at the end of a continuous sideways movement on the display screen 17 then there is a high probability that the driver is not at the display on the display screen 17 is interested, but in an object of the environment, due to the movement of the vehicle behind the display screen 17 wandered. In such a case, it may be appropriate not to complicate the tracking of the object by switching to a highly eye-catching presentation, but to leave the representation unchanged. A switch to the non-transparent state takes place according to this development only when an uneven, with the assumption that the driver pursues an object outside the vehicle with his eyes, incompatible eye movement the driver's gaze on the display screen 17 has led. Paragraph 41 in particular describes making the display particularly transparent as a result of the driver looking away).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising: determining gaze directions of eyes of at least one user, by utilising tracking means; detecting, based on the gaze directions, when the at least one user has not been gazing through a given portion of the heads-up display whereat said image segment of the image is being displayed since at least a predefined time period; and when it is detected that the at least one user has not been gazing through the given portion of the heads-up display since at least the predefined time period, performing any one of: generating the image segment of the image by utilising the at least one rear-view image at a second resolution that is lower than a first resolution, wherein a given image segment of a given image is generated at the first resolution when it is detected that the at least one user has been gazing through the given portion of the heads-up display since at least the predefined time period, or generating an entirety of the image by utilising at least one virtual image as taught by Wyona Lars so as to allow the system to reduce the distraction of the heads-up display when the driver needs to focus on something else.
Claim(s) 7-8 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Ahn et al. US 20220281317 A1 (“Ahn”) in combination with Singhal US 20160229341 A1 (“Singhal”) as applied to claims 1 and 11 above, and further in view of Bongwald US 20150232030 A1 (“Bongwald”).
Regarding Claim 7. Ahn in combination with Singhal teaches the system of claim 1.
Ahn also teaches:
wherein when generating the current image, the at least one processor is configured to:
generate a current image segment of the current image by utilising at least one virtual image (FIG. 1 shows a virtual image in one segment of the current image in which the rear view is displayed [paragraphs 51-58]); and
generate a remainder of the current image by utilising at least one current rear-view image of said region of the surrounding environment (FIG. 1, paragraphs 51-58).
Ahn does not teach:
wherein the at least one processor is configured to:
detect when a reverse gear of the vehicle is activated; and
when it is detected that the reverse gear is activated, generate a current image to be displayed via the heads-up display.
However, Bongwald teaches:
wherein the at least one processor is configured to:
detect when a reverse gear of the vehicle is activated; and
when it is detected that the reverse gear is activated, generate a current image to be displayed via the heads-up display (The display can automatically change to display the rearward view during a reversing maneuver of the vehicle [paragraphs 41-42]).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with wherein the at least one processor is configured to: detect when a reverse gear of the vehicle is activated; and when it is detected that the reverse gear is activated, generate a current image to be displayed via the heads-up display as taught by Bongwald so as to allow the vehicle to automatically display the rear-view image when the vehicle is traveling in reverse.
Regarding Claim 8. Ahn in combination with Singhal and Bongwald teaches the system of claim 7.
Ahn does not teach:
detect that the reverse gear is activated.
However Bongwald teaches:detect that the reverse gear is activated (paragraphs 41-42).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with detect that the reverse gear is activated as taught by Bongwald so as to allow the vehicle to automatically display the rear-view image when the vehicle is traveling in reverse.
Ahn also does not teach:
further comprising at least one side-view camera, wherein the at least one processor is configured to:
capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising the at least one side-view camera; and
when generating the current image, generate the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also.
However, Singhal teaches:
further comprising at least one side-view camera (The cameras at 14 and 16 of FIG. 1A), wherein the at least one processor is configured to:
capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising the at least one side-view camera (FIG. 1A); and
when generating the current image, generate the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also (FIGS. 3A-3B).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising at least one side-view camera, wherein the at least one processor is configured to: capture at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising the at least one side-view camera; and when generating the current image, generate the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also as taught by Singhal so as to allow the vehicle to incorporate not just the location directly behind the vehicle, but the sides behind the vehicle as well, in case an animal or person is about to run behind the vehicle while it is backing up.
Regarding Claim 17. Ahn in combination with Singhal teaches the method of claim 11.
Ahn also teaches:
further comprising:
wherein when generating the current image, performing:
generating a current image segment of the current image by utilising at least one virtual image (FIG. 1 shows a virtual image in one segment of the current image in which the rear view is displayed [paragraphs 51-58]); and wherein when generating the current image, performing:
generating a remainder of the current image by utilising at least one current rear-view image of said region of the surrounding environment (FIG. 1, paragraphs 51-58).
Ahn does not teach:
detecting when a reverse gear of the vehicle is activated; and
when it is detected that the reverse gear is activated, generating a current image to be displayed via the heads-up display.
However, Bongwald teaches:
detecting when a reverse gear of the vehicle is activated; and
when it is detected that the reverse gear is activated, generating a current image to be displayed via the heads-up display (The display can automatically change to display the rearward view during a reversing maneuver of the vehicle [paragraphs 41-42]).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with detecting when a reverse gear of the vehicle is activated; and when it is detected that the reverse gear is activated, generating a current image to be displayed via the heads-up display as taught by Bongwald so as to allow the vehicle to automatically display the rear-view image when the vehicle is traveling in reverse.
Regarding Claim 18. Ahn in combination with Singhal and Bongwald teaches the method of claim 17.
Ahn does not teach:
detect that the reverse gear is activated.
However Bongwald teaches:detect that the reverse gear is activated (paragraphs 41-42).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with detect that the reverse gear is activated as taught by Bongwald so as to allow the vehicle to automatically display the rear-view image when the vehicle is traveling in reverse.
Ahn also does not teach:
further comprising:
capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising at least one side-view camera; and
when generating the current image, generating the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also.
However, Singhal teaches:
further comprising:
capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising at least one side-view camera (The cameras at 14 and 16 of FIG. 1A); and
when generating the current image, generating the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also (FIGS. 3A-3B).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising: capturing at least one side-view image of another region of the surrounding environment that is beside the vehicle, by utilising at least one side-view camera; and when generating the current image, generating the remainder of the current image by utilising the at least one side-view image of the another region of the surrounding environment also as taught by Singhal so as to allow the vehicle to incorporate not just the location directly behind the vehicle, but the sides behind the vehicle as well, in case an animal or person is about to run behind the vehicle while it is backing up.
Claim(s) 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ahn et al. US 20220281317 A1 (“Ahn”) in combination with Singhal US 20160229341 A1 (“Singhal”) as applied to claims 1 and 11 above, and further in view of Sagerian et al. US 20150116507 A1 (“Sagerian”).
Regarding Claim 10. Ahn in combination with Singhal teaches the system of claim 1.
Ahn does not teach:
further comprising at least one temperature sensor and a temperature controlling unit, wherein the at least one processor is configured to:
detect, by utilising the at least one temperature sensor, when a temperature of the heads- up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature; and
when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employ the temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range.
However, Sagerian teaches:
further comprising at least one temperature sensor and a temperature controlling unit (A display system shown in FIG. 1, may include thermal modulation features [paragraph 32], and a temperature sensor at 560 of FIG. 6), wherein the at least one processor is configured to:
detect, by utilising the at least one temperature sensor, when a temperature of the heads- up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature (paragraph 43); and
when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employ the temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range (paragraph 32).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising at least one temperature sensor and a temperature controlling unit, wherein the at least one processor is configured to: detect, by utilising the at least one temperature sensor, when a temperature of the heads- up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature; and when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employ the temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range as taught by Sagerian so as to allow the system to ensure that the display device does not overheat during use.
Regarding Claim 20. Ahn in combination with Singhal teaches the method of claim 11.
Ahn does not teach:
further comprising:
detecting, by utilising at least one temperature sensor, when a temperature of the heads-up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature; and
when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employing a temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range.
However, Sagerian teaches:
further comprising:
detecting, by utilising at least one temperature sensor (A display system shown in FIG. 1, may include thermal modulation features [paragraph 32], and a temperature sensor at 560 of FIG. 6), when a temperature of the heads-up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature (paragraph 43); and
when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employing a temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range (paragraph 32).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Ahn with further comprising: detecting, by utilising at least one temperature sensor, when a temperature of the heads-up display lies outside a predefined operating temperature range of the heads-up display, or an ambient temperature in the surrounding environment is below a predefined threshold temperature; and when it is detected that the temperature of the heads-up display lies outside the predefined operating temperature range, or the ambient temperature is below the predefined threshold temperature, employing a temperature controlling unit to control the temperature of the heads-up display to bring the temperature within the predefined operating temperature range as taught by Sagerian so as to allow the system to ensure that the display device does not overheat during use.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON G CAIN whose telephone number is (571)272-7009. The examiner can normally be reached Monday: 7:30am - 4:30pm EST to Friday 7:30pm - 4:30am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON G CAIN/Examiner, Art Unit 3656
Prosecution Insights