DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Examiner’s note: the restriction is withdrawn in view of the applicant’s remarks. Claims 1-20 are pending for the instant application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 3, 4, 17, 18, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20130162630) in view of HAN et al. (US 20170230647).
Regarding claim 1, KIM discloses A method of displaying a stereoscopic image (KIM, abstract, “A method and apparatus for displaying a stereoscopic image using pixel mapping are provided”), the method comprising:
mapping each sub-pixel among a plurality of sub-pixels of a display panel to a corresponding one of viewpoints to generate a mapping table (KIM, figs.7-9, “[0036] The processor 12 generates pixel mapping data. The pixel mapping data is a table obtained by mapping a viewpoint position of an observer with pixel information on a screen corresponding to the viewpoint position of the observer using barrier information and panel information. [0066] Next, the pixel mapping data generation unit 120 generates pixel mapping data by mapping pixel information corresponding to a sub pixel position of p11 with the viewpoint position of the observer. [0062] First, it is assumed that an observer views a predetermined sub pixel of the panel 100 using a predetermined slit of the barrier 102 in the predetermined viewpoint position A. [0067] Next, the pixel mapping data generation unit 120 stores the generated pixel mapping data in the pixel mapping DB 14. [0068] The pixel mapping data generation process will be described in detail with reference to FIG. 9 based on the implementation example of the parallax barrier scheme of FIG. 7”. Therefore, referring to figs.7 and 9, the repeated performed operations in fig.9 indicate mapping each sub-pixel to a corresponding one of viewpoints to generate the pixel mapping data, aka a mapping table );
generating rendering image data for each of the viewpoints using the mapping table; and generating stereoscopic image data based on the rendering image data for each of the viewpoints (KIM, “[0074] In operation 8200, the apparatus 1 displays, through the display unit 124, stereoscopic image contents in a pixel area on the screen corresponding to the viewpoint position of the observer discerned through the position discernment unit 122 based on the pixel mapping data generated through the pixel mapping data generation unit 120. [0088] When the construction of the pixel mapping data is completed, position information of the observer in a predetermined viewpoint position is discerned, the corresponding pixel mapping DB is searched, and stereoscopic image contents are output to a stereoscopic display based on a pixel mapping table. [0091] through an adaptive scheme for providing optimal stereoscopic image contents in accordance with each of the viewpoint positions of the observer, clear stereoscopic image contents may be displayed”).
On the other hand, KIM fails to explicitly disclose but HAN discloses generating a three-dimensional model (HAN, fig.5A-D, “[0075] the controller 140 may calculate the mixed pixel value by setting a 3D zone in an epipolar domain generated based on an epipolar image configured in the same pixel line of each of the plurality of rendered views, and applying a predetermined weight to the set 3D zone. That is, based on property of the epipolar domain, when the 3D zone is set on the basis of a specific sub-pixel zone of the view of the specific viewpoint selected from the epipolar domain, a zone including the source pixel zone corresponding to each of the view of the specific viewpoint and the view of the adjacent viewpoint may be set”. Therefore, the 3D zone based on the sub-pixels corresponds to the three-dimensional model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined HAN and KIM, to include all limitations of claim 1. That is, applying the 3D zone of HAN to the image rendering of KIM, especially, adding the plurality of adjacent frames of HAN to the generating rendering image data of KIM. The motivation/ suggestion would have been to provide a clear 3D image by generating a multiview image using more received image views than the number of optical views (HAN, [0008]), and calculate the mixed pixel value in consideration of pixel values of other adjacent frames such as a previous frame, a next frame, and the like, rather than simply based on a current frame, and the source pixel zone may be set according to various methods (HAN, [0077])
Regarding claim 2, KIM in view of HAN discloses The method according to claim 1, wherein the three-dimensional model, has been disclosed.
KIM further discloses wherein the stereoscopic image contents is rendered with respect to the sub-pixels mapped to the same viewpoint among the viewpoints (KIM, “[0073] Next, in operation 8100, the apparatus 1 discerns the viewpoint position of the observer through the position discernment unit 122. [0074] In operation 8200, the apparatus 1 displays, through the display unit 124, stereoscopic image contents in a pixel area on the screen corresponding to the viewpoint position of the observer discerned through the position discernment unit 122 based on the pixel mapping data generated through the pixel mapping data generation unit 120”. Therefore, the viewpoint position of the observer discerned corresponds to the same viewpoint).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined HAN and KIM, to include all limitations of claim 2. That is, applying the 3D zone of HAN to the image rendering of KIM. The same motivation of claim 1 applies here.
Regarding claim 17, it recites similar limitations as claim 1, except that it further recites A display system comprising: a main processor and a display device; wherein the display device comprises: a display panel including sub-pixels; lenses overlapping the sub-pixels; and a display panel driver configured to drive the display panel.
Kim further discloses A display system comprising: a main processor and a display device; wherein the display device comprises: a display panel including sub-pixels; lenses overlapping the sub-pixels; and a display panel driver configured to drive the display panel (KIM, fig.3, “[0009] A lenticular scheme in which a lenticular lens plate with a cylindrical lens array vertically arranged thereon is mounted on a front side of an image panel, and a parallax barrier scheme, are representative examples of autostereoscopic schemes. [0013] there is provided an apparatus for displaying a stereoscopic image, including: a stereoscopic image display that includes a panel composed of a plurality of pixels and a parallax barrier that is mounted on a front surface or a rear surface of the panel while being spaced apart from the panel by a predetermined interval; and a processor that generates pixel mapping data obtained by mapping a viewpoint position of an observer with pixel information on a screen corresponding to the viewpoint position of the observer using barrier information and panel information, and outputs stereoscopic image contents in a pixel area on the screen”).
Regarding claim(s) 18, it is interpreted and rejected for the same reasons set forth in claim(s) 2.
Regarding claim 3, KIM in view of HAN discloses The method according to claim 1.
On the other hand, KIM fails to explicitly disclose but HAN discloses wherein the three-dimensional model is rendered with respect to the sub-pixels mapped to at least two adjacent viewpoints among the viewpoints (HAN, “[0109] Next, the generated multiview image is displayed (S830). [0113] in S820 of generating the multiview image, the mixed pixel value may be calculated by setting the 3D zone including a source pixel zone corresponding to each of the selected view of the specific viewpoint and the views of the adjacent viewpoints in the epipolar domain generated based on the epipolar image configured by the same pixel line of each of the plurality of views, and applying a predetermined weight to the set 3D zone”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined HAN and KIM, to include all limitations of claim 3. That is, adding the generated multiview image from adjacent views of HAN to the displaying method of KIM. The motivation/ suggestion would have been to provide a clear 3D image by generating a multiview image using more received image views than the number of optical views (HAN, [0008]).
Regarding claim(s) 19, it is interpreted and rejected for the same reasons set forth in claim(s) 3.
Regarding claim 4, KIM in view of HAN discloses The method according to claim 1.
On the other hand, KIM fails to explicitly disclose but HAN discloses wherein the three-dimensional image data is generated by summing the rendering image data for each of the viewpoints (HAN, “[0109] Next, the generated multiview image is displayed (S830). [0113] in S820 of generating the multiview image, the mixed pixel value may be calculated by setting the 3D zone including a source pixel zone corresponding to each of the selected view of the specific viewpoint and the views of the adjacent viewpoints in the epipolar domain generated based on the epipolar image configured by the same pixel line of each of the plurality of views, and applying a predetermined weight to the set 3D zone”. Therefore, mixing pixel values with a predetermined weight indicates summing the rendering image data). The same motivation of claim 3 applies here.
Claim(s) 5, 6, 11, 12, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20130162630) in view of HAN et al. (US 20170230647), and further in view of Hoppenstein et la. (US 8368690).
Regarding claim 5, KIM in view of HAN discloses The method according to claim 1.
On the other hand, KIM in view of HAN fails to explicitly disclose but Hoppenstein discloses wherein the generating the three-dimensional model generates the three-dimensional model with a first resolution, and the method further comprises up-scaling the rendering image data from the first resolution to a second resolution higher than the first resolution (Hoppenstein, “(53) A plurality of image acquisition devices (e.g., cameras) may be used to capture images from multiple viewing angles, enabling the display of stereoscopic images in a similarly broad angular range to viewers of the display. (71) For example, the cameras may each provide image data in 960.times.540 format while the display panel has a native resolution of 1920.times.1080. In such cases, the M camera images may be interlaced at the resolution of the camera, i.e., 960 columns in this example. The resolution of the interlaced image is changed to match that of the display panel. In a preferred embodiment, a bicubic interpolation is used to change the resolution to determine the value and position of an interpolated pixel. The interpolated image is then scaled to match the size of the display panel”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Hoppenstein and KIM in view of HAN, to include all limitations of claim 5. That is, applying the changing resolution of Hoppenstein to the 3D model of KIM in view of HAN. The motivation/ suggestion would have been enabling autostereoscopic 3D and 4D viewing of images (Hoppenstein, (7)).
Regarding claim(s) 20, it is interpreted and rejected for the same reasons set forth in claim(s) 5.
Regarding claim 6, KIM in view of HAN and Hoppenstein discloses The method according to claim 5.
On the other hand, KIM in view of HAN fails to explicitly disclose but Hoppenstein discloses wherein the second resolution is a resolution of the display panel (Hoppenstein, “(71) For example, the cameras may each provide image data in 960.times.540 format while the display panel has a native resolution of 1920.times.1080. In such cases, the M camera images may be interlaced at the resolution of the camera, i.e., 960 columns in this example. The resolution of the interlaced image is changed to match that of the display panel. In a preferred embodiment, a bicubic interpolation is used to change the resolution to determine the value and position of an interpolated pixel. The interpolated image is then scaled to match the size of the display panel”). The same motivation of claim 5 applies here.
Regarding claim(s) 11, 12, they are covered by claim(s) 5, 6, thus is/are rejected under similar rationale as claim(s) 5, 6, respectively
Claim(s) 7, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20130162630) in view of HAN et al. (US 20170230647), Hoppenstein et la. (US 8368690), and further in view of CAI et al. (US 20220399101).
Regarding claim 7, KIM in view of HAN and Hoppenstein discloses The method according to claim 5.
On the other hand, KIM in view of HAN and Hoppenstein fails to explicitly disclose but CAI discloses wherein the up-scaling is performed through a super resolution algorithm utilizing deep learning (CAI, “[0052] For instance, as in FIG. 4, the super-resolution method may be based on a U-net architecture. In another instance, the super-resolution method may be based on a convolutional neural network super-resolution method selected from the group including but not limited to Super-Resolution Convolutional Neural Network (SRCNN), Fast Super-Resolution Convolutional Neural Network (FSRCNN), and Very Deep Super Resolution (VDSR). [0056] Each step in the upsampling path consists of an upsampling of the feature map followed by a 2×2 convolution that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3×3 convolutions, each followed by a ReLU”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined CAI into the combination of Hoppenstein and KIM, HAN, to include all limitations of claim 7. That is, applying the upsampling such as FSRCNN of CAI to the image data of KIM, HAN, and Hoppenstein. The motivation/ suggestion would have been enhancing computed tomography image resolution (CAI, [0005]).
Regarding claim(s) 13, it is covered by claim(s) 7, thus is/are rejected under similar rationale as claim(s) 7.
Claim(s) 8, 9, 14, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20130162630) in view of HAN et al. (US 20170230647), Hoppenstein et al. (US 8368690), and further in view of LIU et al. (US 20230057261) and Liu2 et al. (US 20100265313).
Regarding claim 8, KIM in view of HAN and Hoppenstein discloses The method according to claim 5.
On the other hand, KIM in view of HAN and Hoppenstein fails to explicitly disclose but LIU discloses wherein the rendering image data for a moving image is up-scaled through an ultra high speed up-scaling algorithm (LIU, “[0107] At 962, an up-sampling submodule 620 is used to up-sample the motion information of the first inter frame (i.e. MV map 216) to generate up-sampled motion information, also of dimensions sH×sW×2. In some examples, the up-sampling submodule 620 uses an interpolation technique, such as bicubic interpolation, to up-sample the motion information”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined LIU into the combination of Hoppenstein and KIM, HAN. That is, applying the up-sample the motion information of LIU to the image data of KIM, HAN, and Hoppenstein. The motivation/ suggestion would have been generating a super-resolution version of a compressed video stream (LIU, [0002]).
On the other hand, KIM in view of HAN, Hoppenstein and LIU fails to explicitly disclose but Liu2 discloses the rendering image data for a still image is up-scaled through a high definition up-scaling algorithm (Liu2, “[0065] FIG. 4 is a flow diagram of steps for automatically generating high-definition panoramic video in response to still and video image capture combined with super-resolution up-scaling techniques”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined LIU2 into the combination of Hoppenstein and KIM, HAN and LIU. That is, applying the up-sample the still image of Liu2 to the image data of KIM, HAN, Hoppenstein, and LIU. The motivation/ suggestion would have been high definition video provides not only a higher resolution, but also extends the horizontal frame, thus being a form of frame-by-frame panorama. This embodiment of the invention is particularly applicable when the video camera used does not have an imaging device with the desired aspect ratio (e.g., sufficient width) (Liu2, [0065]).
Regarding claim 9, KIM in view of HAN, Hoppenstein, LIU, and Liu2 discloses The method according to claim 8.
On the other hand, KIM in view of HAN, Liu2 fails to explicitly disclose but Hoppenstein discloses wherein the ultra high speed up-scaling algorithm utilizes bicubic interpolation (Hoppenstein, “(71) For example, the cameras may each provide image data in 960.times.540 format while the display panel has a native resolution of 1920.times.1080. In such cases, the M camera images may be interlaced at the resolution of the camera, i.e., 960 columns in this example. The resolution of the interlaced image is changed to match that of the display panel. In a preferred embodiment, a bicubic interpolation is used to change the resolution to determine the value and position of an interpolated pixel. The interpolated image is then scaled to match the size of the display panel”). The same motivation of claim 5 applies here.
Furthermore, LIU also discloses wherein the ultra high speed up-scaling algorithm utilizes bicubic interpolation (LIU, “[0107] At 962, an up-sampling submodule 620 is used to up-sample the motion information of the first inter frame (i.e. MV map 216) to generate up-sampled motion information, also of dimensions sH×sW×2. In some examples, the up-sampling submodule 620 uses an interpolation technique, such as bicubic interpolation, to up-sample the motion information”). The same motivation of combining LIU in claim 8 applies here.
Regarding claim(s) 14, 15, they are covered by claim(s) 8, 9, thus is/are rejected under similar rationale as claim(s) 8, 9, respectively.
Claim(s) 10, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over KIM et al. (US 20130162630) in view of HAN et al. (US 20170230647), Hoppenstein et al. (US 8368690), LIU et al. (US 20230057261) and Liu2 et al. (US 20100265313), and further in view of CAI et al. (US 20220399101).
Regarding claim 10, KIM in view of HAN, Hoppenstein, LIU, and Liu2 discloses The method according to claim 8.
On the other hand, KIM in view of HAN, Hoppenstein, LIU, and Liu2 fails to explicitly disclose but CAI discloses wherein the high definition up-scaling algorithm utilizes a fast super resolution convolutional neural network (CAI, “[0052] For instance, as in FIG. 4, the super-resolution method may be based on a U-net architecture. In another instance, the super-resolution method may be based on a convolutional neural network super-resolution method selected from the group including but not limited to Super-Resolution Convolutional Neural Network (SRCNN), Fast Super-Resolution Convolutional Neural Network (FSRCNN), and Very Deep Super Resolution (VDSR). [0056] Each step in the upsampling path consists of an upsampling of the feature map followed by a 2×2 convolution that halves the number of feature channels, a concatenation with the correspondingly cropped feature map from the contracting path, and two 3×3 convolutions, each followed by a ReLU”). The same motivation of claim 7 applies here.
Regarding claim(s) 16, it is covered by claim(s) 7, thus is/are rejected under similar rationale as claim(s) 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRACE Q LI whose telephone number is (571)270-0497. The examiner can normally be reached Monday - Friday, 8:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DEVONA FAULK can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRACE Q LI/Primary Examiner, Art Unit 2618 1/9/2026