DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Application claims priority to foreign application with application number JP2022-006403 dated 01/19/2022. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The information disclosures statements dated 01/22/2026 and 04/27/2026 have been considered and placed in the application file.
Response to Arguments
Applicant’s arguments, see pg. 10, filed 04/27/2026, with respect to the rejection of claim 14 under 35 U.S.C. 101 have been fully considered and are persuasive. The rejection of claim 14 under 35 U.S.C. 101 has been withdrawn.
Applicant’s arguments with respect to the rejection of the amended claims 1, 13, and 14, incorporating limitations of the previous claim 5, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments with respect to the rejection of the amended claim 11 have been fully considered but they are not persuasive. Applicant states that claim 11 is amended similarly to claim 1, but it is not.
Claim Objections
Claims 1, 13, and 14 objected to because of the following informalities: The phrase "a simultaneously display state" seems grammatically incorrect. The specification refers to "a simultaneous display state", which is likely the intended phrasing. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang et al. (US 20220155595 A1, hereinafter "Wang").
Regarding claim 11, Okuda teaches: A head-mounted display (pg. 5 “The stereoscopic image display device 2 is a device capable of displaying an image signal, such as a display monitor, a TV receiver, or a head-mounted display.”) comprising:
a display panel (pg. 15 “The stereoscopic image display device 2 sequentially displays the screen data received from the stereoscopic image display control device 1 on the screen of the display 24; fig. 4 shows headset display panel);
a stereo camera configured to capture a stereo video to be displayed as the display images (pg. 14 “In this way, the compound eye camera 50 can generate the stereoscopic image data required to display a stereoscopic image. In particular, in this embodiment, the compound eye camera 50 (stereoscopic video recording circuit 53) adds management information, including information on the distance between the left and right cameras 51, 52 when capturing the stereoscopic video data and information on the focal length at the time of capturing the data, to the stereoscopic video data and records the data as a stereoscopic video stream on a recording medium.”); and
alternately generate either a left-eye display image or a right-eye display image for a frame of a video (pg. 13-14 “When capturing a stereoscopic image with the compound eye camera 50, the left and right cameras 51 and 52 alternately capture still images. In this way, the compound eye camera 50 sequentially generates a left eye image taken by the left eye camera 51 and a right eye image taken by the right eye camera 52, which are necessary to display a stereoscopic image. For example, a left eye image 61 and a right eye image 62 as shown in FIG. 5 are generated.”);
control a display panel to display either the left-eye display image or the right-eye display image (pg. 15 “Specifically, as shown in FIG. 6(b), the stereoscopic image display control device 1 alternately outputs screen data representing an image for the left eye (left eye image) and an image for the right eye (right eye image) to the stereoscopic image display device 2. The stereoscopic image display device 2 sequentially displays the screen data received from the stereoscopic image display control device 1 on the screen of the display 24.”).
Okuda does not explicitly teach: a display control device comprising:
one or more processors; and
one or more memories storing instructions, that, when executed by the one or more processors, cause the one or more processors to:
control a display panel comprising a left region and a right region to display either the left-eye display image or the right-eye display image in a corresponding one of the left region or the right region; and
control the display panel to suppress the other one of the left region or the right region from displaying an image.
Wang teaches: a display control device comprising:
one or more processors (fig. 3 processor 302); and
one or more memories storing instructions ([0019] “The head mounted display may include: a first display screen and a second display screen, configured to display an image; one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the head mounted display is enabled to perform the image display method in any possible design of the foregoing aspects.”), that, when executed by the one or more processors, cause the one or more processors to:
control a display panel comprising a left region and a right region to display either the left-eye display image or the right-eye display image in a corresponding one of the left region or the right region; and control the display panel to suppress the other one of the left region or the right region from displaying an image ([0145] “In another embodiment, the HMD may alternately display the left-eye image and the right-eye image in different display cycles T according to a preset display frame rate f. For example, referring to FIG. 12, the display frame rate f is 120 fps, and the HMD may sequentially perform displaying according to the following order: displaying the first frame of left-eye image on the left display screen in the first display cycle T; displaying the second frame of right-eye image on the right display screen in the second display cycle T; displaying a third frame of left-eye image on the left display screen in the third display cycle T; . . . ; displaying a 120.sup.th frame of right-eye image on the right display screen in a 120.sup.th display cycle T.”;
fig. 12 shows timing where only one eye image is displayed at a time).
Okuda and Wang are analogous to the claimed invention because they are in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda with the teachings of Wang to replace the active shutter-based system of Okuda with a system with a separate display for each eye. The motivation would have been to increase reliability by reducing dependence on additional electro-chemical components.
Regarding claim 12, the combination of Okuda in view of Wang teaches: The head-mounted display according to claim 11, wherein the stereo camera is configured to:
alternately capture an image from a left viewpoint and an image from a right viewpoint of the frame of the stereo video (Okuda pg. 13-14 “When capturing a stereoscopic image with the compound eye camera 50, the left and right cameras 51 and 52 alternately capture still images.”); and
supply the images to the display control device (Okuda pg. 14 “The compound eye camera 50 records the left eye image and right eye image thus generated as stereoscopic image data on a recording medium (for example, an optical disk, a memory card, or a hard disk) using a stereoscopic image recording circuit 53. In this way, the compound eye camera 50 can generate the stereoscopic image data required to display a stereoscopic image.
In particular, in this embodiment, the compound eye camera 50 (stereoscopic video recording circuit 53) adds management information, including information on the distance between the left and right cameras 51, 52 when capturing the stereoscopic video data and information on the focal length at the time of capturing the data, to the stereoscopic video data and records the data as a stereoscopic video stream on a recording medium.”).
Claim(s) 1, 2, 3, 13, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi et al. (US 20160173867 A1; hereinafter "Ichihashi").
Regarding claim 1, Okuda teaches:
alternately generate either a left-eye display image or a right-eye display image for a frame of a video (pg. 13-14 “When capturing a stereoscopic image with the compound eye camera 50, the left and right cameras 51 and 52 alternately capture still images. In this way, the compound eye camera 50 sequentially generates a left eye image taken by the left eye camera 51 and a right eye image taken by the right eye camera 52, which are necessary to display a stereoscopic image. For example, a left eye image 61 and a right eye image 62 as shown in FIG. 5 are generated.”); and
control a display panel comprising a left region and a right region to display at least one of the left-eye display image or the right-eye display image in a corresponding one of the left region or the right region (pg. 15 “Specifically, as shown in FIG. 6(b), the stereoscopic image display control device 1 alternately outputs screen data representing an image for the left eye (left eye image) and an image for the right eye (right eye image) to the stereoscopic image display device 2. The stereoscopic image display device 2 sequentially displays the screen data received from the stereoscopic image display control device 1 on the screen of the display 24.”).
Okuda does not explicitly teach: A display control device comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to:
control a display panel comprising a left region and a right region to display at least one of the left-eye display image or the right-eye display image in a corresponding one of the left region or the right region.
Wang teaches: A display control device comprising:
one or more processors (fig. 3 processor 302); and
one or more memories storing instructions ([0019] “The head mounted display may include: a first display screen and a second display screen, configured to display an image; one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the head mounted display is enabled to perform the image display method in any possible design of the foregoing aspects.”), that, when executed by the one or more processors, cause the one or more processors to:
control a display panel comprising a left region and a right region to display at least one of the left-eye display image or the right-eye display image in a corresponding one of the left region or the right region (fig. 12, [0145] “In another embodiment, the HMD may alternately display the left-eye image and the right-eye image in different display cycles T according to a preset display frame rate f. For example, referring to FIG. 12, the display frame rate f is 120 fps, and the HMD may sequentially perform displaying according to the following order: displaying the first frame of left-eye image on the left display screen in the first display cycle T; displaying the second frame of right-eye image on the right display screen in the second display cycle T; displaying a third frame of left-eye image on the left display screen in the third display cycle T; . . . ; displaying a 120.sup.th frame of right-eye image on the right display screen in a 120.sup.th display cycle T.”).
Okuda and Wang are analogous to the claimed invention because they are in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda with the teachings of Wang to replace the active shutter-based system of Okuda with a system with a separate display for each eye. The motivation would have been to increase reliability by reducing dependence on additional electro-chemical components.
The combination of Okuda in view of Wang does not explicitly teach: gradually decrease brightness of either the left-eye display image or the right-eye display image being displayed and gradually increase brightness of the other display image during a simultaneously display state in which the left-eye display image and the right-eye display image are simultaneously displayed.
Li teaches: decrease brightness of either the left-eye display image or the right-eye display image being displayed and increase brightness of the other display image during a simultaneously display state in which the left-eye display image and the right-eye display image are simultaneously displayed ([0039] “It can be seen by analyzing the histograms of FIGS. 4(a) and 4(b) that the average value of the brightness of the left-eye image is higher, and thus it can be determined that the left-eye image is brighter. In this case, the difference of the brightness between the two images can be reconciled by simply increasing the brightness of the right-eye image, or decreasing the brightness of the left-eye image. Alternatively, both the brightness of the left-eye image and that of the right-eye image may be adjusted to an average value of the brightness of the two images to achieve the same effect.”).
Li is analogous to the claimed invention because it is in the same field of stereoscopic 3D displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang with the teachings of Li in order to correct for differences in brightness between the captured left and right images, as described in the background of Li.
The combination of Okuda in view of Wang and Li does not explicitly teach: gradually decrease brightness of either the left-eye display image or the right-eye display image being displayed and gradually increase brightness of the other display image during a simultaneously display state in which the left-eye display image and the right-eye display image are simultaneously displayed.
Ichihashi teaches a gradual change in brightness of either the left-eye display or the right-eye display ([0025] “When switching from the first to the second mode, controller 104 gradually decreases the luminance of pixels 112 for right-eye, as indicated by the solid line in FIG. 4.”;
[0022] explains that the first mode is a simultaneous display mode and the second mode is a single-eye mode.)
Ichihashi is analogous to the claimed invention because it is in the same field of stereoscopic 3D displays, in particular switching between display modes. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang and Li with the teachings of Ichihashi to perform gradual adjustments in brightness for individual eye displays. The motivation would have been for the comfort of the user, as described in Ichihashi [0029].
Regarding claim 2, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 1, wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
acquire data regarding a stereo video photographed by a stereo camera from left and right viewpoints (Okuda pg. 13-14 “When capturing a stereoscopic image with the compound eye camera 50, the left and right cameras 51 and 52 alternately capture still images.”); and
alternately generate either the left-eye display image or the right-eye display image by alternately using a left-viewpoint photographed image and a right-viewpoint photographed image of a frame of the stereo video (Okuda pg. 13-14 “In this way, the compound eye camera 50 sequentially generates a left eye image taken by the left eye camera 51 and a right eye image taken by the right eye camera 52, which are necessary to display a stereoscopic image. For example, a left eye image 61 and a right eye image 62 as shown in FIG. 5 are generated.”).
Regarding claim 3, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 2, wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
alternately acquire the left-viewpoint photographed image and the right-viewpoint photographed image (Okuda pg. 13-14 “When capturing a stereoscopic image with the compound eye camera 50, the left and right cameras 51 and 52 alternately capture still images. In this way, the compound eye camera 50 sequentially generates a left eye image taken by the left eye camera 51 and a right eye image taken by the right eye camera 52, which are necessary to display a stereoscopic image. For example, a left eye image 61 and a right eye image 62 as shown in FIG. 5 are generated.”), and
generate the display images by using the acquired photographed images (Okuda pg. 14 “The compound eye camera 50 records the left eye image and right eye image thus generated as stereoscopic image data on a recording medium (for example, an optical disk, a memory card, or a hard disk) using a stereoscopic image recording circuit 53. In this way, the compound eye camera 50 can generate the stereoscopic image data required to display a stereoscopic image.
In particular, in this embodiment, the compound eye camera 50 (stereoscopic video recording circuit 53) adds management information, including information on the distance between the left and right cameras 51, 52 when capturing the stereoscopic video data and information on the focal length at the time of capturing the data, to the stereoscopic video data and records the data as a stereoscopic video stream on a recording medium.”).
Regarding claims 13 and 14, they are rejected with the same references, rationale, and motivation to combine as claim 1 because their limitations substantially correspond to the limitations of claim 1, as well as the additional limitation in claim 14 of: A non-transitory computer-readable medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform operations (Wang [0019] “The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the head mounted display is enabled to perform the image display method in any possible design of the foregoing aspects.”;
[0088] “In addition, the memory 303 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, and universal flash storage (universal flash storage, UFS).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the additional teachings of Okuda to use non-transitory memory for more permanent storage capabilities.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) as applied to claim 2 above, and further in view of Sadi et al. (US 20160088287 A1, hereinafter "Sadi").
Regarding claim 4, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 2, but does not explicitly teach: wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
store, for each pixel, an integrated correction amount of a plurality of kinds of corrections necessary to generate the display images from the photographed images; and
generate the display images by correcting the photographed images by the integrated correction amount.
Sadi teaches: wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
store, for each pixel, an integrated correction amount of a plurality of kinds of corrections necessary to generate the display images from the photographed images ([0096] “In particular embodiments, cameras 112 may have some lens distortion as well as some deviation relative to a target position or orientation 114. In particular embodiments, corrections for these effects may be static, and they may be pre-calibrated and corrected using lookup tables in the front end.”); and
generate the display images by correcting the photographed images by the integrated correction amount ([0096] “As an example and not by way of limitation, panorama leveling, vignette correction, lens distortion correcting, white balance correction, exposure correction and matching, or viewpoint adjustment may be applied directly to an image. In this manner, an image may be operated on before any compression-induced color or feature shifts take place, which may reduce the occurrence of visible correction artifacts.”).
Sadi is analogous to the claimed invention because it is in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the teachings of Sadi to add an integrated system to compensate for lens distortion and other aberrations. The motivation would have been to ensure the output image looks as realistic as possible.
Claim(s) 5, 6, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) as applied to claims 1, 2, and 13 above, and further in view of Rockel et al. (US 20220101593 A1, hereinafter "Rockel").
Regarding claim 5, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 1, wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
switching between a simultaneous display state and a one-side display state in which the either one of the left-eye display image and the right-eye display image is displayed (Wang fig. 2 teaches a simultaneous display state, fig. 12 teaches a one-sided display state, [0147]-[0148] “In some other embodiments, the HMD may switch between different display solutions. For example, by default, the HMD may perform displaying by using the solution shown in FIG. 2. When the peak current is greater than the rated current, the HMD may prompt, by using a voice, vibration, display prompt information shown in FIG. 14, or the like, the user whether to switch the display solution. After detecting an operation that the user chooses to switch the display solution, the HMD may display an image by using the display solution for reducing the peak current provided in this embodiment of this application.”).
The combination of Okuda in view of Wang, Li, and Ichihashi does not explicitly teach: gradually decrease brightness of a pre-switching display image and concurrently increase brightness of a post-switching display image
Rockel teaches: gradually decrease brightness of a pre-switching display image and concurrently increase brightness of a post-switching display image ([0335] “…the computer system gradually blurs out and/or darkens the portions of the representation of the physical environment that are still visible, and replaces them with virtual content (e.g., expansion of the existing virtual content, adding new virtual content, etc.). In some embodiments, the computer system displays the virtual content, such as virtual wallpaper, virtual room decor, virtual scenery, virtual movie screen, virtual desktop, etc., which gradually replaces the blurred and/or darkened portions of the representation of the physical environment (e.g., fading in from behind the portions of the representation of the physical environment, or creeping in from surrounding regions of the portions of the representation of the physical environment, etc.). When the transition is completed, the user's field of view of the first computer-generated experience has been expanded and less of the physical environment is visible via the display generation component.”).
Rockel is analogous to the claimed invention because it is in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the teachings of Rockel to add the ability to smoothly fade between display modes and to apply this type of transition to the display modes of the aforementioned invention. The motivation would have been to improve the user experience by adding aesthetically pleasing features.
Regarding claim 6, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 2, wherein,
the instructions that, when executed by the one or more processors, further cause the one or more processors to switch the simultaneous display state to the one-side display state (Wang fig. 2 teaches a simultaneous display state, fig. 12 teaches a one-sided display state, [0147]-[0148] “In some other embodiments, the HMD may switch between different display solutions. For example, by default, the HMD may perform displaying by using the solution shown in FIG. 2. When the peak current is greater than the rated current, the HMD may prompt, by using a voice, vibration, display prompt information shown in FIG. 14, or the like, the user whether to switch the display solution. After detecting an operation that the user chooses to switch the display solution, the HMD may display an image by using the display solution for reducing the peak current provided in this embodiment of this application.”).
The combination of Okuda in view of Wang, Li, and Ichihashi does not explicitly teach that this occurs in display switching from a content image to each photographed image.
Rockel teaches: display switching from a content image to each photographed image ([0335] “In some embodiments, the computer system displays the virtual content, such as virtual wallpaper, virtual room decor, virtual scenery, virtual movie screen, virtual desktop, etc., which gradually replaces the blurred and/or darkened portions of the representation of the physical environment (e.g., fading in from behind the portions of the representation of the physical environment, or creeping in from surrounding regions of the portions of the representation of the physical environment, etc.). When the transition is completed, the user's field of view of the first computer-generated experience has been expanded and less of the physical environment is visible via the display generation component.”).
Rockel is analogous to the claimed invention because it is in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the teachings of Rockel to switch to the one-sided alternating display in the specified situation; the motivation would have been to reduce heat and power consumption, as taught by Wang.
Regarding claim 16, it is rejected with the same references, rationale, and motivation to combine as claim 5 because its limitations substantially correspond to the limitations of claim 5.
Claim(s) 7, 18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) as applied to claims 1, 13, and 14 above, and further in view of Loferer et al. (US 20200158498 A1, hereinafter "Loferer").
Regarding claim 7, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 1, as well as switching between the simultaneous display state and the one-side display state (Wang, see claim 5).
The combination of Okuda in view of Wang, Li, and Ichihashi does not explicitly teach: wherein, in switching between the simultaneous display state and the one-side display state, the instructions that, when executed by the one or more processors, further cause the one or more processors to change a pixel value on data of the display images with time in accordance with a rule of linearly changing display brightness with time.
Loferer teaches: …change a pixel value on data of the display images with time in accordance with a rule of linearly changing display brightness with time (fig. 14, [0014] describes the use of gamma correction to adjust the brightness scale of a display device to appear linear to the human eye; [0090] “FIG. 14 depicts the gamma of a commercially available monitor 1 (full line). With a suitably chosen correcting function (long broken line) for the brightness display on monitor 1 it is possible to adjust a linear brightness impression on the resulting image (short broken line).”).
Loferer is analogous to the claimed invention because it pertains to the same issue of correcting a digital image output for a digital display, particularly when adjusting brightness, as discussed by Okuda in view of Wang, Li, and Ichihashi. Furthermore, the use of gamma curves to adjust display brightness is a well-known concept in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the teachings of Loferer to implement a gamma curve to ensure that fade-in/fade-out transitions appear smooth and linear to a user. The motivation would have been to improve the user experience by adding aesthetically pleasing features.
Regarding claims 18 and 20, they are rejected with the same references, rationale, and motivation to combine as claim 7 because their limitations substantially correspond to the limitations of claim 7.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) as applied to claim 1 above, and further in view of Inada et al. (US 20160227203 A1, hereinafter "Inada").
Regarding claim 8, the combination of Okuda in view of Wang, Li, and Ichihashi teaches: The display control device according to claim 1, but does not explicitly teach: wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
render an additional image indicating additional information directly in a region within a predetermined range from a center of each display image to which an inverse distortion has been given in view of a distortion of an eyepiece lens during a viewing time, after giving a distortion according to a position to the additional image.
Inada teaches: render an additional image indicating additional information ([0089] “FIG. 9B shows exemplary images where auxiliary data is displayed.”; [0091] “(1) After main data images are rendered from the left and right views, rectangular auxiliary data images are additionally rendered at the very front of the two images and subjected to distortion correction.”) directly in a region within a predetermined range from a center of each display image to which an inverse distortion has been given in view of a distortion of an eyepiece lens during a viewing time, after giving a distortion according to a position to the additional image ([0092] “If the image for the flat display 16 (shown in the right subfigures) is extracted from the images for the HMD 18 generated by this technique, inverse distortion correction becomes progressively ineffective toward the image edges. The initial rectangular captured images thus become increasingly difficult to reconstitute at their edges. In view of these characteristics, in particular, if technique (1) is employed, the auxiliary data images are preferably displayed around the center of the main data image. This reduces the possibility of the extracted auxiliary data image partially dropping.”).
Inada and the combination of Okuda in view of Wang, Li, and Ichihashi are analogous to the claimed invention because they are in the same field of stereoscopic 3D head-mounted displays. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi with the teachings of Inada to add the ability to overlay an additional image over the displayed 3D image. The motivation would be to display important information or warnings about a user’s 3D capture area, as taught by Inada.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) and further in view of Inada (US 20160227203 A1) as applied to claim 8 above, and further in view of Zhou et al. (CN 1996389 A, hereinafter "Zhou").
Regarding claim 9, the combination of Okuda in view of Wang, Li, and Ichihashi and further in view of Inada teaches: The display control device according to claim 8, but does not explicitly teach: wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
store, as model data for the additional image, data regarding a plane including a plurality of polygons each having at least one vertex disposed in an inner region of the plane; and
render the additional image having a distortion for each of the polygons on a basis of position coordinates of the vertex defined on a plane of each display image.
Zhou teaches: store, as model data for the additional image, data regarding a plane including a plurality of polygons each having at least one vertex disposed in an inner region of the plane ([0009] “1.1. Set target 1. The target is a two-dimensional plane, and black rectangles are arranged in a matrix on the target plane, with a quantity of 4 to 100. The length and width of the rectangles are in the range of 3 to 50 mm. The vertices of the black rectangles on the target surface are selected as feature points, and the number of feature points ranges from 16 to 400. There must be at least one straight line on the target plane, which includes three or more collinear feature points whose coordinates are unknown.”;
[0010] 1.2 Within the effective field of view of the camera, place the target freely at least once. Take an image at each position, which is called a distortion calibration image. All feature points on the target should be included in the captured image. All captured target images are superimposed together, and the superimposed target image should fill the entire image as much as possible.
[0011] 1.3 Extract the actual image coordinates of feature points in all distortion calibration images, and mark feature points with at least 3 collinear features as the same group of feature points;
[0012] 1.4. Utilizing the property that feature points in the same group should be collinear in the distortion calibration image, an optimization function is established with the goal of minimizing the sum of the distances from the ideal image coordinates to the fitted straight line. The camera distortion coefficient k is estimated through a nonlinear optimization method.
[0013] 1.5 Save the distortion coefficient k to the system parameter file for later use in distortion correction.”); and
render the additional image having a distortion for each of the polygons on a basis of position coordinates of the vertex defined on a plane of each display image ([0014] “After obtaining the distortion coefficients of the camera, distortion correction can be performed on the image.”).
Zhou and the combination of Okuda in view of Wang and further in view of Inada are analogous to the claimed invention because they pertain to the same issue of lens distortion correction for a digital display. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang and further in view of Inada with the teachings of Zhou to implement the claimed method of matching and calibrating the distortion correction of an overlaid image. The motivation would be to improve the user experience by ensuring that the information or warning images taught by Inada are clear and appropriately configured for viewing.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Okuda (WO 2011024423 A1) in view of Wang (US 20220155595 A1), Li (US 20120105595 A1), and Ichihashi (US 20160173867 A1) and further in view of Inada (US 20160227203 A1) as applied to claim 8 above, and further in view of Sylvan et al. (US 20170372457 A1, hereinafter "Sylvan").
Regarding claim 10, the combination of Okuda in view of Wang, Li, and Ichihashi and further in view of Inada teaches: The display control device according to claim 8, but does not explicitly teach: wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to:
exclude a region in each display image that is to be hidden by the additional image from rendering.
Sylvan teaches: exclude a region in each display image that is to be hidden by the additional image from rendering ([0063] “Generating the composite image overlaying each layer, including the visual data layer, the vector graphic layer, and the graphical user interface overlay on top of one another. In one example, the signed distance field includes depth values for each pixel represented by the signed distance field. Thus, in this example, any reprojected data of the signed distance field having a depth value that is behind a corresponding pixel of the reprojected rendered visual scene data may be determined to be occluded and consequently not rendered in the composite image.”).
Sylvan and the combination of Okuda in view of Wang, Li, and Ichihashi and further in view of Inada are analogous to the claimed invention because they are in the same field of graphical rendering for a head-mounted display. Furthermore, the concept of occlusion culling is well known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Okuda in view of Wang, Li, and Ichihashi and further in view of Inada with the teachings of Sylvan to not render scene geometry which is hidden by a superimposed image. The motivation would have been to improve efficiency by avoiding unnecessary computation.
Allowable Subject Matter
Claims 15, 17, and 19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
The prior art of record does not teach or suggest the limitation: change brightness alternately in either the left-eye display image or the right-eye display image displayed in the simultaneous display state, for each frame, before changing brightness in the other image. The prior art teaches independently changing brightness in left-eye and right-eye display images as discussed in the rejection of claim 1, but not on an alternating frame-by-frame basis as specified in claims 15, 17, and 19.
References Cited
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mao (US 20220222026 A1) teaches a head-mounted display which is able to switch between a normal display mode in which both eye images are displayed simultaneously and a low power consumption display mode in which left eye and right eye images are displayed alternately.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN STATZ whose telephone number is (571)272-6654. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENJAMIN TOM STATZ/ Examiner, Art Unit 2611
/TAMMY GODDARD/ Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights