DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed January 29, 2026 has been entered. Claims 1, 3-13, 15 and 17-20 remain pending in the application. Applicant’s amendments to the Claims have overcome each and every objection previously set forth in the Non-Final Office Action mailed December 2, 2025.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 4-5 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Bruls (US 2017/0085859) in view of He et al. (US 2025/0039352) and in further view of Wang et al. (CN115002438).
Regarding claim 1, Bruls discloses an image display method, adapted to a 3D display system comprising a 3D display device (paragraph 0039: “3D display device 60”), and the method comprising: obtaining a first image and a second image by splitting an input image according to a 3D image format (paragraph 0071: “unit SPH 601 for splitting the 2D frame in the input signal into two (or more) 3D subframes according to a first 3D format”); determining whether the input image is a 3D format image complying with the 3D image format (paragraphs 0012-0013: “setting the 3D status signal based on an assessment of the respective format scores to indicate the format of the video signal…the 3D status being either a 2D status or a 3D status indicating one format of a set of possible 3D video formats”) by performing a stereo matching process on the first image and the second image (paragraphs 0012, 0122-0123: “determining respective format scores for a number of the possible 3D formats by processing the video data according to respective predetermined format properties for deriving and comparing the respective 3D subframes” which may comprise of “calculating a disparity estimation between the 3D subframes”); in response to determining that the input image is the 3D format image complying with the 3D image format (paragraph 0013: “Based on the assessment one of the 3D formats may have a reliably high score, and then the 3D status signal is correspondingly set to indicate the format of the video signal”), determining to deliver the input image to a runtime (paragraphs 0013, 0049: “provides the 3D status signal for controlling a 3D video display, i.e. to set the operational mode for correctly rendering the video signal…in 3D mode” where the 3D mode is a runtime); and in response to determining that the input image is not the 3D format image complying with the 3D image format (paragraph 0013: “If none of the 3D formats has a sufficiently high score, a 2D video signal is assumed and the status is correspondingly set”), determining not to deliver the input image to the runtime (paragraphs 0013, 0049: “provides the 3D status signal for controlling a 3D video display, i.e. to set the operational mode for correctly rendering the video signal…in 2D mode” where delivering the video to the 2D runtime implies not delivering the video to the 3D runtime). However, Bruls fails to explicitly disclose determining to deliver the input image to a runtime complying with a OpenXR standard to enable an image interweaving process to perform the image interweaving process on the input image to generate an interweaving image, and displaying the interweaving image through the 3D display device, wherein the runtime is developed according to the OpenXR standard and hardware characteristics of the 3D display device; and determining not to deliver the input image to the runtime complying with the OpenXR standard to disable the image interweaving process.
In the related art of 3D display devices, He discloses determining to deliver the input image to a runtime (He paragraph 0015: “In 102, when the naked-eye 3D display device is switched to a 3D display mode”) to enable an image interweaving process to perform the image interweaving process on the input image to generate an interweaving image (He paragraph 0015: “two or more view pixels of the stereoscopic image are set to be interleaved on the display panel”), and displaying the interweaving image through the 3D display device (He paragraph 0013: “The naked-eye 3D display device is configured to display a stereoscopic image”); and determining not to deliver the input image to the runtime (He paragraph 0014: “In 101, when the naked-eye 3D display device is switched to a 2D display mode”) to disable the image interweaving process (He FIG. 1, paragraphs 0014-0015: in the 2D display mode, the step in the 3D display mode where “two or more view pixels of the stereoscopic image are set to be interleaved on the display panel” is clearly omitted). A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. More specifically, Bruls’ 2D and 3D display modes can be realized via a finite number of predictable solutions in the prior art. It would have been obvious to try He’s 3D display method and device to execute Bruls’ 2D and 3D display modes. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bruls to incorporate the teachings of He to lead to the anticipated success of displaying a 3D image via an image interweaving process and to reduce 3D crosstalk while allowing for switching between 2D and 3D display modes (He paragraph 0007). However, Bruls, modified by He, still fails to explicitly disclose a runtime complying with a OpenXR standard, wherein the runtime is developed according to the OpenXR standard and hardware characteristics of the 3D display device.
In the related art of 3D display, Wang discloses a runtime (Wang paragraph n0110: “the XR preview module needs to integrate the SDK (Software Development Kit) of the target XR device and use the same rendering process as the runtime of the target XR device”) complying with a OpenXR standard, wherein the runtime is developed according to the OpenXR standard (Wang paragraph n0110: “use the OpenXR standard to integrate the runtime on the PC platform of each XR device, and use the left and right swapchains created by OpenXR in the XR preview module rendering process to achieve binocular display”) and hardware characteristics of the 3D display device (Wang paragraph n0110: “adapts to the distortion effect of the connected target XR device” where the distortion effect is incorporated as a display parameter of the target XR device; thus, the runtime is developed according to the device’s display parameters which depend on hardware characteristics of the device, see also paragraph n0079). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Wang to enable a runtime to work across a wide variety of devices since OpenXR is compatible with most mainstream XR devices (Wang paragraph n0110).
Regarding claim 4, Bruls, modified by He and Wang, discloses the image display method claimed in claim 1, wherein determining whether the input image is the 3D format image complying with the 3D image format by performing the stereo matching process on the first image and the second image comprises: performing the stereo matching process on the first image and the second image to generate a disparity map of the first image and the second image (Bruls paragraphs 0082-0083: “DE/MC 604 may be arranged for…calculating a disparity estimation between the 3D subframes and compensating at least one of the 3D subframes based on the disparity estimation before further comparing”); calculating a matching number of a plurality of first pixels in the first image and a plurality of second pixels in the second image according to the disparity map (Bruls paragraphs 0072, 0075: “the frame L is compared to frame L' by the correspondence calculation unit MAD [605]” where “The mean absolute difference of each (or a subset of) corresponding pixels in the 2 parts is calculated” and “The output of the correspondence calculation is converted in scoring unit CMP 607 to a format score 610”); and determining whether the input image is the 3D format image complying with the 3D image format according to the matching number (Bruls paragraph 0073: “Format scores for different 3D formats are to be assessed to determine the actual 3D format used, if any”).
Regarding claim 5, Bruls, modified by He and Wang, discloses the image display method claimed in claim 4, wherein determining whether the input image is the 3D format image complying with the 3D image format according to the matching number comprises: determining whether the matching number meets a matching condition (Bruls paragraph 0073: “The format scores may be compared to each other and/or to respective thresholds”); in response to the fact that the matching number meets the matching condition, determining that the input image is the 3D format image complying with the 3D image format (Bruls paragraph 0074: “if the score is sufficiently reliable, taking a decision that the first 3D format is present at the input”); and in response to the fact that the matching number does not meet the matching condition, determining that the input image is not the 3D format image complying with the 3D image format (Bruls paragraph 0073: “If none of the 3D formats has a sufficient confidence level, a 2D signal (mono video) is assumed”).
Regarding claim 10, it is the corresponding system (comprising a storage device and a processer) configured to execute the method claimed in claim 1. Therefore, Bruls, modified by He and Wang, discloses the limitations of claim 10 as it does the limitations of claim 1.
Claim(s) 3 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, He and Wang in view of Han et al. (US 2015/0350626).
Regarding claim 3, Bruls discloses an image display method, adapted to a 3D display system comprising a 3D display device (Bruls paragraph 0039: “3D display device 60”), and the method comprising: obtaining a first image and a second image by splitting an input image according to a 3D image format (Bruls paragraph 0071: “unit SPH 601 for splitting the 2D frame in the input signal into two (or more) 3D subframes according to a first 3D format”); determining whether the input image is a 3D format image complying with the 3D image format (Bruls paragraphs 0012-0013: “setting the 3D status signal based on an assessment of the respective format scores to indicate the format of the video signal…the 3D status being either a 2D status or a 3D status indicating one format of a set of possible 3D video formats”) by performing a stereo matching process on the first image and the second image (Bruls paragraphs 0012, 0122-0123: “determining respective format scores for a number of the possible 3D formats by processing the video data according to respective predetermined format properties for deriving and comparing the respective 3D subframes” which may comprise of “calculating a disparity estimation between the 3D subframes”); in response to determining that the input image is the 3D format image complying with the 3D image format (Bruls paragraph 0013: “Based on the assessment one of the 3D formats may have a reliably high score, and then the 3D status signal is correspondingly set to indicate the format of the video signal”), determining to deliver the input image to a runtime (Bruls paragraphs 0013, 0049: “provides the 3D status signal for controlling a 3D video display, i.e. to set the operational mode for correctly rendering the video signal…in 3D mode” where the 3D mode is a runtime). However, Bruls fails to explicitly disclose determining to deliver the input image to a runtime complying with a OpenXR standard to enable an image interweaving process to perform the image interweaving process on the input image to generate an interweaving image, and displaying the interweaving image through the 3D display device, wherein the runtime is developed according to the OpenXR standard and hardware characteristics of the 3D display device; and in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image, enabling the image interweaving process to be performed on the synthetic image to generate the interweaving image, and the interweaving image is displayed through the 3D display device.
In related art, He discloses determining to deliver the input image to a runtime (He paragraph 0015: “In 102, when the naked-eye 3D display device is switched to a 3D display mode”) to enable an image interweaving process to perform the image interweaving process on an image to generate an interweaving image (He paragraph 0015: “two or more view pixels of the stereoscopic image are set to be interleaved on the display panel”), and displaying the interweaving image through the 3D display device (He paragraph 0013: “The naked-eye 3D display device is configured to display a stereoscopic image”). A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. More specifically, Bruls’ 2D and 3D display modes can be realized via a finite number of predictable solutions in the prior art. It would have been obvious to try He’s 3D display method and device to execute Bruls’ 2D and 3D display modes. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bruls to incorporate the teachings of He to lead to the anticipated success of displaying a 3D image via an image interweaving process and to reduce 3D crosstalk while allowing for switching between 2D and 3D display modes (He paragraph 0007). However, Bruls, modified by He, still fails to explicitly disclose a runtime complying with a OpenXR standard, wherein the runtime is developed according to the OpenXR standard and hardware characteristics of the 3D display device; and in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image.
In related art, Wang discloses a runtime (Wang paragraph n0110: “the XR preview module needs to integrate the SDK (Software Development Kit) of the target XR device and use the same rendering process as the runtime of the target XR device”) complying with a OpenXR standard, wherein the runtime is developed according to the OpenXR standard (Wang paragraph n0110: “use the OpenXR standard to integrate the runtime on the PC platform of each XR device, and use the left and right swapchains created by OpenXR in the XR preview module rendering process to achieve binocular display”) and hardware characteristics of the 3D display device (Wang paragraph n0110: “adapts to the distortion effect of the connected target XR device” where the distortion effect is incorporated as a display parameter of the target XR device; thus, the runtime is developed according to the device’s display parameters which depend on hardware characteristics of the device, see also paragraph n0079). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Wang to enable a runtime to work across a wide variety of devices since OpenXR is compatible with most mainstream XR devices (Wang paragraph n0110). However, Bruls, modified by He and Wang, still fails to explicitly disclose in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image.
In the related art of displaying 3D images, Han discloses in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image (Han paragraph 0017: “when the input image is the 2D image, converting the input 2D image to a 3D image; and displaying the converted 3D image by changing the display mode to the 3D mode”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Han to use the 3D mode more easily without the user having to recognize whether the input image is the 2D image or the 3D image (Han paragraph 0012).
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, He and Wang in view of Jin et al. (US 2016/0048837).
Regarding claim 6, Bruls, modified by He and Wang, discloses the image display method claimed in claim 5, wherein determining whether the matching number meets the matching condition comprises: determining whether the matching ratio (Bruls paragraph 0072: “The output of the correspondence calculation is converted in scoring unit CMP 607 to a format score 610 for the SBS 3D format, e.g. by normalizing”) is greater than a threshold value (Bruls paragraph 0013: “the format scores are assessed, e.g. compared to a predetermined threshold”), wherein if the matching ratio is greater than the threshold value, the matching number meets the matching condition (Bruls paragraph 0013: “Based on the assessment one of the 3D formats may have a reliably high score, and then the 3D status signal is correspondingly set to indicate the format of the video signal”); and if the matching ratio is not greater than the threshold value, the matching number does not meet the matching condition (Bruls paragraph 0013: “If none of the 3D formats has a sufficiently high score, a 2D video signal is assumed”). However, Bruls fails to explicitly disclose calculating a matching ratio of the matching number to a pixel number of the first image. In the related art of image matching, Jin discloses calculating a matching ratio of the matching number to a pixel number of the first image (Jin paragraph 0058: “When the number or proportion of successfully matched pixel points reaches a predetermined threshold (e.g., the ratio of the number of successfully matched pixel points in the total number of all pixel points of the UnionPay logo image exceeds 70%)”). Using a ratio measure is a known technique to condense information into a manageable metric for analysis and comparison across various contexts. One of ordinary skill in the art would have been capable of applying calculation of a ratio to an image matching method to simplify data interpretation and enable comparison across different entities. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Jin to yield the predictable result of determining whether the matching number meets the matching condition in various contexts, e.g., when images have different sizes.
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, He and Wang in view of Song et al. (US 2013/0120536).
Regarding claim 7, Bruls, modified by He and Wang, discloses the image display method claimed in claim 4. However, Bruls fails to explicitly disclose the disparity map comprises a plurality of valid disparity values and a plurality of invalid disparity values, and the matching number is a number of the valid disparity values. In the related art of disparity estimation, Song discloses the disparity map (Song paragraph 0008: “A depth image (e.g. a disparity map) may be produced”) comprises a plurality of valid disparity values (Song paragraph 0025: “A number of valid disparity values may be determined from said disparity map”) and a plurality of invalid disparity values (Song paragraph 0025: “there will be a low number of invalid disparity values”), and the matching number is a number of the valid disparity values (Song paragraphs 0025-0026: “The number of valid disparity values may be compared with the threshold value and it is decided whether there is a sufficient number of valid disparity values” where the number of valid disparity values represents the correspondence between the two partial images). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Song to reduce the data quantity to be processed and enable quicker processing of the data (Song paragraph 0010).
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, He and Wang in view of Akao (US 2014/0218357).
Regarding claim 8, Bruls, modified by He and Wang, discloses the image display method claimed in claim 4. However, Bruls fails to disclose taking a first image block with a first target pixel point on the first image as a center; calculating a plurality of similarity degrees between the first image block and a plurality of second image blocks on the second image, wherein a Y-axis position of the first image block is the same as a Y-axis position of the second image block; and obtaining a valid disparity value or an invalid disparity value corresponding to the first target pixel point on the disparity map according to the similarity degree respectively corresponding to the second image blocks. In the related art of parallax estimation, Akao discloses taking a first image block with a first target pixel point on the first image as a center (Akao Fig. 2, paragraph 0140: “a pixel of interest LP = (5, 3) of the pixel region (block) 121 of the L image”); calculating a plurality of similarity degrees between the first image block and a plurality of second image blocks on the second image (Akao paragraphs 0136-0143: a known step in the “block matching-based parallax information acquisition process”, as evidenced by supporting patent application Matono et al. (US 2015/0310621) – see Conclusion; Akao Fig. 2: the dotted lines in the R image indicate a plurality of second image blocks are compared to the first image block for matching), wherein a Y-axis position of the first image block is the same as a Y-axis position of the second image block (Akao Fig. 2: the blocks in both the L and R images have Y-axis position = 3); and obtaining a valid disparity value or an invalid disparity value (this limitation is disclosed in an alternative clause and thus, read only on the first limitation) corresponding to the first target pixel point on the disparity map (Akao paragraphs 0140-0143: “parallax d(5, 3) between the L and R images of the pixel position (x, y)=(5, 3) of the L image is computed”) according to the similarity degree respectively corresponding to the second image blocks (Akao paragraph 0139: “a block similar to the selected block is detected in the R image”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Akao to generate an optimum multi-perspective image based on reliability of the estimated parallax (Akao paragraph 0051).
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, He, Wang and Akao in view of Song.
Regarding claim 9, Bruls, modified by He, Wang and Akao, discloses the image display method claimed in claim 8, wherein obtaining the valid disparity value or the invalid disparity value corresponding to the first target pixel point on the disparity map according to the similarity degree respectively corresponding to the second image blocks comprises: if a second target image block matching a first target image block among the second image blocks is obtained according to the similarity degree (Akao paragraph 0139: “a block similar to the selected block is detected in the R image”), obtaining the valid disparity value corresponding to the first target pixel point on the disparity map based on an X-axis position of a second target pixel point centered on the second target image block and an X-axis position of the first target pixel point (Akao Fig. 2, paragraphs 0140-0143: “Parallax d(5, 3)=(7, 3)-(5, 3)=(2, 0)”). However, Bruls fails to disclose if the second target image block matching the first target image block among the second image blocks is not obtained according to the similarity degree, obtaining the invalid disparity value corresponding to the first target pixel point on the disparity map. In related art, Song discloses if the second target image block matching the first target image block among the second image blocks is not obtained according to the similarity degree, obtaining the invalid disparity value corresponding to the first target pixel point on the disparity map (Song paragraph 0025: “If therefore a lens of the serial camera 1 is masked or soiled, no valid disparity values can be determined in at least one area of said partial image because for pixels in this area it is not possible to find the respective pixel in the other partial image. As a result, there will be a low number of invalid disparity values”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Song to reduce the data quantity to be processed and enable quicker processing of the data (Song paragraph 0010).
Claim(s) 11-12, 15, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Bruls in view of Song and in further view of Jin.
Regarding claim 11, Bruls discloses an image display method, adapted to a 3D display system comprising a 3D display device (Bruls paragraph 0039: “3D display device 60”), and the method comprising: obtaining a first image and a second image by splitting an input image according to a 3D image format (Bruls paragraph 0071: “unit SPH 601 for splitting the 2D frame in the input signal into two (or more) 3D subframes according to a first 3D format”); performing a stereo matching process on the first image and the second image to generate a disparity map of the first image and the second image (Bruls paragraphs 0082-0083: “DE/MC 604 may be arranged for…calculating a disparity estimation between the 3D subframes and compensating at least one of the 3D subframes based on the disparity estimation before further comparing”); calculating a matching number of a plurality of first pixels in the first image and a plurality of second pixels in the second image according to the disparity map (Bruls paragraphs 0072, 0075: “the frame L is compared to frame L' by the correspondence calculation unit MAD [605]” where “The mean absolute difference of each (or a subset of) corresponding pixels in the 2 parts is calculated” and “The output of the correspondence calculation is converted in scoring unit CMP 607 to a format score 610”); and determining whether the input image is a 3D format image complying with the 3D image format according to the matching number (Bruls paragraph 0073: “Format scores for different 3D formats are to be assessed to determine the actual 3D format used, if any”); and in response to determining that the input image is the 3D format image complying with the 3D image format (Bruls paragraph 0013: “Based on the assessment one of the 3D formats may have a reliably high score, and then the 3D status signal is correspondingly set to indicate the format of the video signal”), switching the 3D display device to operate in a 3D stereoscopic display mode according to the 3D image format (Bruls paragraphs 0013, 0049: “provides the 3D status signal for controlling a 3D video display, i.e. to set the operational mode for correctly rendering the video signal…in 3D mode”), wherein determining whether the input image is the 3D format image complying with the 3D image format according to the matching number comprises: determining whether the matching number meets a matching condition (Bruls paragraph 0073: “The format scores may be compared to each other and/or to respective thresholds”), wherein determining whether the matching number meets the matching condition comprises: determining whether the matching ratio (Bruls paragraph 0072: “The output of the correspondence calculation is converted in scoring unit CMP 607 to a format score 610 for the SBS 3D format, e.g. by normalizing”) is greater than a threshold value (Bruls paragraph 0013: “the format scores are assessed, e.g. compared to a predetermined threshold”), wherein if the matching ratio is greater than the threshold value, the matching number meets the matching condition (Bruls paragraph 0013: “Based on the assessment one of the 3D formats may have a reliably high score, and then the 3D status signal is correspondingly set to indicate the format of the video signal”); and if the matching ratio is not greater than the threshold value, the matching number does not meet the matching condition (Bruls paragraph 0013: “If none of the 3D formats has a sufficiently high score, a 2D video signal is assumed”). However, Bruls fails to explicitly disclose the matching number is a number of valid disparity values; and calculating a matching ratio of the matching number to a pixel number of the first image, wherein the matching ratio is obtained by dividing the number of valid disparity values by the pixel number of the first image.
In related art, Song discloses the matching number is a number of valid disparity values (Song paragraphs 0025-0026: “The number of valid disparity values may be compared with the threshold value and it is decided whether there is a sufficient number of valid disparity values” where the number of valid disparity values represents the correspondence between the two partial images). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Bruls to incorporate the teachings of Song to reduce the data quantity to be processed and enable quicker processing of the data (Song paragraph 0010). However, Bruls, modified by Song, still fails to explicitly disclose calculating a matching ratio of the matching number to a pixel number of the first image, wherein the matching ratio is obtained by dividing the matching number by the pixel number of the first image.
In related art, Jin discloses calculating a matching ratio of the matching number to a pixel number of the first image, wherein the matching ratio is obtained by dividing the matching number by the pixel number of the first image (Jin paragraph 0058: “When the number or proportion of successfully matched pixel points reaches a predetermined threshold (e.g., the ratio of the number of successfully matched pixel points in the total number of all pixel points of the UnionPay logo image exceeds 70%)”). Using a ratio measure is a known technique to condense information into a manageable metric for analysis and comparison across various contexts. One of ordinary skill in the art would have been capable of applying calculation of a ratio to an image matching method to simplify data interpretation and enable comparison across different entities. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Jin to yield the predictable result of determining whether the matching number meets the matching condition in various contexts, e.g., when images have different sizes.
Regarding claim 12, Bruls, modified by Song and Jin, discloses the image display method claimed in claim 11, wherein the method further comprises: in response to determining that the input image is not the 3D format image complying with the 3D image format (Bruls paragraph 0013: “If none of the 3D formats has a sufficiently high score, a 2D video signal is assumed and the status is correspondingly set”), switching the 3D display device to operate in a 2D display mode (Bruls paragraphs 0013, 0049: “provides the 3D status signal for controlling a 3D video display, i.e. to set the operational mode for correctly rendering the video signal…in either 2D or 3D mode”).
Regarding claim 15, Bruls, modified by Song and Jin, discloses the image display method claimed in claim 11, wherein in response to the fact that the matching number meets the matching condition, determining that the input image is the 3D format image complying with the 3D image format (Bruls paragraph 0074: “if the score is sufficiently reliable, taking a decision that the first 3D format is present at the input”); and in response to the fact that the matching number does not meet the matching condition, determining that the input image is not the 3D format image complying with the 3D image format (Bruls paragraph 0073: “If none of the 3D formats has a sufficient confidence level, a 2D signal (mono video) is assumed”).
Regarding claim 17, Bruls, modified by Song and Jin, discloses the image display method claimed in claim 11, wherein the disparity map (Song paragraph 0008: “A depth image (e.g. a disparity map) may be produced”) comprises a plurality of valid disparity values (Song paragraph 0025: “A number of valid disparity values may be determined from said disparity map”) and a plurality of invalid disparity values (Song paragraph 0025: “there will be a low number of invalid disparity values”).
Regarding claim 20, it is the corresponding system (comprising a storage device and a processer) configured to execute the method claimed in claim 11. Therefore, Bruls, modified by Song and Jin, discloses the limitations of claim 20 as it does the limitations of claim 11.
Claim(s) 13 is rejected under 35 U.S.C. 103 as being unpatentable over Bruls, Song and Jin in view of Han.
Regarding claim 13, Bruls, modified by Song and Jin, discloses the image display method claimed in claim 11, wherein the method further comprises: switching the 3D display device to operate in the 3D stereoscopic display mode according to the 3D image format (as claimed in claim 11). However, Bruls fails to disclose in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image. In related art, Han discloses in response to determining that the input image is not the 3D format image complying with the 3D image format, generating a synthetic image complying with the 3D image format based on the input image (Han paragraph 0017: “when the input image is the 2D image, converting the input 2D image to a 3D image; and displaying the converted 3D image by changing the display mode to the 3D mode”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Han to use the 3D mode more easily without the user having to recognize whether the input image is the 2D image or the 3D image (Han paragraph 0012).
Claim(s) 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Bruls, Song and Jin in view of Akao.
Regarding claim 18, Bruls, modified by Song and Jin, discloses the image display method claimed in claim 11. However, Bruls fails to disclose taking a first image block with a first target pixel point on the first image as a center; calculating a plurality of similarity degrees between the first image block and a plurality of second image blocks on the second image, wherein a Y-axis position of the first image block is the same as a Y-axis position of the second image block; and obtaining a valid disparity value or an invalid disparity value corresponding to the first target pixel point on the disparity map according to the similarity degree respectively corresponding to the second image blocks. In related art, Akao discloses taking a first image block with a first target pixel point on the first image as a center (Akao Fig. 2, paragraph 0140: “a pixel of interest LP = (5, 3) of the pixel region (block) 121 of the L image”); calculating a plurality of similarity degrees between the first image block and a plurality of second image blocks on the second image (Akao paragraphs 0136-0143: a known step in the “block matching-based parallax information acquisition process”, as evidenced by supporting patent application Matono et al. (US 2015/0310621) – see Conclusion; Akao Fig. 2: the dotted lines in the R image indicate a plurality of second image blocks are compared to the first image block for matching), wherein a Y-axis position of the first image block is the same as a Y-axis position of the second image block (Akao Fig. 2: the blocks in both the L and R images have Y-axis position = 3); and obtaining a valid disparity value or an invalid disparity value (this limitation is disclosed in an alternative clause and thus, read only on the first limitation) corresponding to the first target pixel point on the disparity map (Akao paragraphs 0140-0143: “parallax d(5, 3) between the L and R images of the pixel position (x, y)=(5, 3) of the L image is computed”) according to the similarity degree respectively corresponding to the second image blocks (Akao paragraph 0139: “a block similar to the selected block is detected in the R image”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Bruls to incorporate the teachings of Akao to generate an optimum multi-perspective image based on reliability of the estimated parallax (Akao paragraph 0051).
Regarding claim 19, Bruls, modified by Song, Jin and Akao, discloses the image display method claimed in claim 18, wherein obtaining the valid disparity value or the invalid disparity value corresponding to the first target pixel point on the disparity map according to the similarity degree respectively corresponding to the second image blocks comprises: if a second target image block matching a first target image block among the second image blocks is obtained according to the similarity degree (Akao paragraph 0139: “a block similar to the selected block is detected in the R image”), obtaining the valid disparity value corresponding to the first target pixel point on the disparity map based on an X-axis position of a second target pixel point centered on the second target image block and an X-axis position of the first target pixel point (Akao Fig. 2, paragraphs 0140-0143: “Parallax d(5, 3)=(7, 3)-(5, 3)=(2, 0)”); and if the second target image block matching the first target image block among the second image blocks is not obtained according to the similarity degree, obtaining the invalid disparity value corresponding to the first target pixel point on the disparity map (Song paragraph 0025: “If therefore a lens of the serial camera 1 is masked or soiled, no valid disparity values can be determined in at least one area of said partial image because for pixels in this area it is not possible to find the respective pixel in the other partial image. As a result, there will be a low number of invalid disparity values”).
Response to Arguments
Applicant's arguments with respect to independent claim 1 have been fully considered but they are not persuasive.
Regarding the argument that “the 2D/3D display modes disclosed in Bruls and He pertain to hardware-level or display-circuit-level operating states, whereas the "runtime complying with an OpenXR standard" recited in Claim 1 refers to a software execution environment that is selectively invoked to perform image processing, including image interleaving”, Bruls teaches controlling a 3D video display to operate either in a 3D mode or a 2D mode according to the detected video signal format. Bruls additionally teaches the invention can be implemented as computer software running on one or more data processors and/or digital signal processors (Bruls paragraph 0130) and a processor is arranged for processing and generating the image data included in the display signal to be transferred and displayed on a display device (Bruls paragraphs 0046-0049). Therefore, the software running on the processor(s) can act as an execution environment that provides an interface between the application (e.g., input unit 51) and the underlying hardware (e.g., display device 60). The 2D and 3D modes in Bruls can be thought of as two different runtimes in which specific image processing operations are selectively enabled and performed.
Furthermore, MPEP 2111.01 Plain Meaning sections I, II, and III disclose the words of a claim must be given their “plain meaning” unless such meaning is inconsistent with the specification, it is improper to import claim limitations from the specification, and “plain meaning” refers to the ordinary and customary meaning given to the term by those of ordinary skill in the art, respectively. Under broadest reasonable interpretation, “a runtime complying with a OpenXR standard to enable an image interweaving process” encompasses a software execution environment that is involved in making an image interweaving process occur. The image interweaving process itself does not necessarily need to occur within the software execution environment and instead, could occur at a hardware level. Therefore, the 2D/3D hardware-level display modes in He can be interpreted to be controlled by separate runtimes. For example, delivering the input image to one runtime evokes hardware interleaving of the left and right picture pixels, while delivering the input image to the other runtime does not.
Thus, Bruls teaches a control flow of determining whether to deliver a video signal to a 3D display mode runtime based on the input video signal format. He teaches the 3D display mode runtime can enable an image interleaving process. Wang teaches the 3D display device’s runtime can be implemented in compliance with the OpenXR standard. The combination of Bruls, He and Wang teaches selectively delivering the input image to a runtime complying with the OpenXR standard based on a 3D format determination, to enable or disable an image interleaving process.
Applicant's arguments with respect to independent claim 11 have been fully considered but they are not persuasive.
Regarding the argument that “Bruls does not utilize, after generating disparity information, the number of valid disparity values to determine whether an input image complies with a 3D image format”, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Examiner acknowledges that Bruls fails to teach the matching number is the number of valid disparity values. Examiner relies on the combination of Bruls with Song to teach the missing limitations (as detailed in the above rejection of claim 11). In view of the combination of Bruls and Song, one of ordinary skill in the art could reach the conclusion that instead of using the format score as calculated in Bruls to determine the 3D image format, the number of valid disparity values can be used to determine the 3D image format.
Regarding the argument that “Bruls teaches away from retaining or explicitly counting invalid disparity values, because the presence of such invalid values would undermine the core purpose of Bruls, namely, reliable format detection based on intensity-based comparison”, Bruls discloses calculating a correspondence between the 3D subframes using each or a subset of corresponding pixels in the 2 parts (Bruls paragraph 0072). Only corresponding pixels of the subframes are used in the correspondence calculation and even a subset of the corresponding pixels can be used. Thus, Bruls does not necessarily rely on dense disparity estimation, as argued by the Applicant, since pixels in the first image with no corresponding pixels in the second image, as in the case of invalid disparity values, are simply not used in the correspondence calculation.
Regarding the argument that “the proposed combination of the valid-disparity-based analysis of Song with the MAD-based format detection of Bruls, as asserted in the Office Action, does not constitute a reasonable or non-hindsight modification of the cited references. The teachings of Bruls and Song address disparity estimation in fundamentally different technical contexts and for different purposes”, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Furthermore, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). Bruls teaches calculating a correspondence between 3D subframes that is converted to a format score. Additionally, Bruls teaches other correspondences between the 3D subframes may be estimated besides a MAD calculation (Bruls paragraph 0072). Song teaches using the number of valid disparity values to represent correspondence between two partial images (Song paragraphs 0025-0026). Thus, the combined teachings of Bruls and Song would have suggested to those of ordinary skill in the art at the time that instead of using the format score as calculated in Bruls to represent the correspondence between two subframes, the number of valid disparity values can be used to indicate the level of matching.
Regarding the argument that “Normalization in Bruls therefore operates on the magnitude of the error metric itself, and does not alter the underlying basis of the correspondence calculation. That is, a person having ordinary skill in the art would not have been motivated to substitute the normalization step of Bruls with the pixel-ratio-based calculation of Jin, because doing so would depart from the manner in which normalization is defined, applied, and functionally integrated in Bruls”, Bruls teaches other correspondences between the 3D subframes may be estimated, such as calculating a correlation between the 3D subframes (Bruls paragraph 0072). The normalization in Bruls is not confined to scaling error-based metrics as argued. Furthermore, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). Bruls teaches calculating a matching number representing the correspondence between two subframes to determine the 3D format. Song teaches the correspondence between two partial images can be represented by the number of valid disparity values, in which valid disparity values are disparity values for successfully matching pixels. Jin teaches calculating a ratio of the number of successfully matched pixels to the total number of pixels in the image. The combined teachings of Bruls, Song and Jin would have suggested to those of ordinary skill in the art that instead of using the format score as calculated in Bruls to represent the correspondence between two subframes, the ratio of valid disparity values can be used to indicate the level of matching.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE ZHAO whose telephone number is (703)756-5986. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.Z./Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights