DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
1. The information disclosure statement (IDS) submitted on 7/3/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 102
2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
3. Claims 1-3, 9, 12-15, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Cheng (US-2014/0254917-A1).
4. As per claim 1, Cheng discloses: A binocular image generation method, comprising: (Cheng, [0007], “According to a second aspect of the present invention, an exemplary auto-convergence method is disclosed. The exemplary auto-convergence method includes ... generating an output stereo image pair for playback.”)
obtaining an original image;
determining, based on depth values of a plurality of pixels in a salient region of the original image, (Cheng, [0022], “The disparity unit 102 is arranged for performing a disparity analysis upon the input stereo image pair, and accordingly obtaining a disparity distribution DD of the input stereo image pair. For example, the disparity unit 102 may employ one of a stereo matching algorithm, a feature point extraction and matching algorithm, and a region-based motion estimation algorithm to get the statistical analysis of the disparity distribution DD.” and [0003], “In general, the disparity of an object/pixel presented in a stereo image pair composed of a left-view image and a right-view image determines user's depth perception of the object/pixel.”) a target depth value corresponding to a zero-disparity plane in a binocular image to be generated; and (Cheng, [0023], “It is assumed that the any user would feel most comfortable when pixels are displayed with zero disparity. Hence, the convergence unit 104 would check the disparity distribution DD to find the disparity value with a highest occurrence frequency (i.e., the disparity value to which a largest accumulation number corresponds) in the input stereo image pair. For example, the disparity value D3 shown in FIG. 2 corresponds to the highest occurrence frequency.” and [0022], “For example, the disparity unit 102 may employ one of a stereo matching algorithm, a feature point extraction and matching algorithm, and a region-based motion estimation algorithm to get the statistical analysis of the disparity distribution DD. Please refer to FIG. 2, which is a histogram diagram of the disparity distribution DD of the input stereo image pair. As shown in FIG. 2, the disparity values derived from the left-view image IMG_L and the right-view image IMG_R are within a disparity range delimited by D1 and D2, where D2 is the largest positive disparity and D1 is the smallest negative disparity.”; Examiner’s note: Pixels at zero disparity are in the same plane.)
generating the binocular image based on the target depth value and depth values of a plurality of pixels in the original image. (Cheng, [0006], “According to a first aspect of the present invention, an exemplary auto-convergence system is disclosed. The exemplary auto-convergence system includes a disparity unit, a convergence unit and an active learning unit. The disparity unit is arranged for performing a disparity analysis upon an input stereo image pair, and accordingly obtaining a disparity distribution of the input stereo image pair. The convergence unit is coupled to the disparity unit, and arranged for adaptively adjusting the input stereo image pair according to the disparity distribution and a learned convergence range, and accordingly generating an output stereo image pair for playback.” and [0023], “It is assumed that the any user would feel most comfortable when pixels are displayed with zero disparity. Hence, the convergence unit 104 would check the disparity distribution DD to find the disparity value with a highest occurrence frequency (i.e., the disparity value to which a largest accumulation number corresponds) in the input stereo image pair. For example, the disparity value D3 shown in FIG. 2 corresponds to the highest occurrence frequency. The convergence position adjustment made by the convergence unit 104 thus regards the disparity value D3 as an initial zero disparity position to determine a shifted disparity range based on the original disparity range derived from the disparity distribution DD, and then compares the shifted disparity range with the learned convergence range CR to check if the shifted disparity range should be further shifted to make the initial zero disparity position shifted/changed to a final zero disparity position.”)
5. As per claim 2, Cheng discloses: The method according to claim 1, wherein the determining, based on depth values of a plurality of pixels in a salient region of the original image, a target depth value corresponding to a zero-disparity plane in a binocular image to be generated comprises: (See rejection for claim 1.)
generating a first histogram based on the depth values of the plurality of pixels in the salient region of the original image; and (Cheng, Figure 2; [0022], “The disparity unit 102 is arranged for performing a disparity analysis upon the input stereo image pair, and accordingly obtaining a disparity distribution DD of the input stereo image pair. For example, the disparity unit 102 may employ one of a stereo matching algorithm, a feature point extraction and matching algorithm, and a region-based motion estimation algorithm to get the statistical analysis of the disparity distribution DD. Please refer to FIG. 2, which is a histogram diagram of the disparity distribution DD of the input stereo image pair. As shown in FIG. 2, the disparity values derived from the left-view image IMG_L and the right-view image IMG_R are within a disparity range delimited by D1 and D2, where D2 is the largest positive disparity and D1 is the smallest negative disparity.”)
determining, based on a distribution of the depth values in the first histogram, the target depth value corresponding to the zero-disparity plane in the binocular image to be generated. (Cheng, [0023], “It is assumed that the any user would feel most comfortable when pixels are displayed with zero disparity. Hence, the convergence unit 104 would check the disparity distribution DD to find the disparity value with a highest occurrence frequency (i.e., the disparity value to which a largest accumulation number corresponds) in the input stereo image pair. For example, the disparity value D3 shown in FIG. 2 corresponds to the highest occurrence frequency. The convergence position adjustment made by the convergence unit 104 thus regards the disparity value D3 as an initial zero disparity position to determine a shifted disparity range based on the original disparity range derived from the disparity distribution DD, and then compares the shifted disparity range with the learned convergence range CR to check if the shifted disparity range should be further shifted to make the initial zero disparity position shifted/changed to a final zero disparity position.” and [0022], “Please refer to FIG. 2, which is a histogram diagram of the disparity distribution DD of the input stereo image pair.” and [0006], “According to a first aspect of the present invention, an exemplary auto-convergence system is disclosed. The exemplary auto-convergence system includes a disparity unit, a convergence unit and an active learning unit. The disparity unit is arranged for performing a disparity analysis upon an input stereo image pair, and accordingly obtaining a disparity distribution of the input stereo image pair. The convergence unit is coupled to the disparity unit, and arranged for adaptively adjusting the input stereo image pair according to the disparity distribution and a learned convergence range, and accordingly generating an output stereo image pair for playback.”)
6. As per claim 3, Cheng discloses: The method according to claim 2, wherein the determining, based on a distribution of the depth values in the first histogram, the target depth value corresponding to the zero-disparity plane in the binocular image to be generated comprises: (See rejection for claim 2.)
determining the target depth value corresponding to the zero-disparity plane in the binocular image to be generated, based on a depth value range in the first histogram within which a largest number of pixels are distributed. (Cheng, [0023], “It is assumed that the any user would feel most comfortable when pixels are displayed with zero disparity. Hence, the convergence unit 104 would check the disparity distribution DD to find the disparity value with a highest occurrence frequency (i.e., the disparity value to which a largest accumulation number corresponds) in the input stereo image pair. For example, the disparity value D3 shown in FIG. 2 corresponds to the highest occurrence frequency. The convergence position adjustment made by the convergence unit 104 thus regards the disparity value D3 as an initial zero disparity position to determine a shifted disparity range based on the original disparity range derived from the disparity distribution DD, and then compares the shifted disparity range with the learned convergence range CR to check if the shifted disparity range should be further shifted to make the initial zero disparity position shifted/changed to a final zero disparity position.” and [0022], “The disparity unit 102 is arranged for performing a disparity analysis upon the input stereo image pair, and accordingly obtaining a disparity distribution DD of the input stereo image pair. For example, the disparity unit 102 may employ one of a stereo matching algorithm, a feature point extraction and matching algorithm, and a region-based motion estimation algorithm to get the statistical analysis of the disparity distribution DD. Please refer to FIG. 2, which is a histogram diagram of the disparity distribution DD of the input stereo image pair.”)
7. As per claim 9, Cheng discloses: The method according to claim 1, wherein the original image is a video frame; and after the generating the binocular image, the method further comprises:
generating a stereoscopic video based on a plurality of binocular images. (Cheng, [0001], “The disclosed embodiments of the present invention relate to the stereo video/image playback ...” and [0023], “Specifically, the convergence unit 104 is coupled to the disparity unit 102 and the active learning unit 106, and arranged for adaptively adjusting the input stereo image pair according to the disparity distribution DD and a learned convergence range CR, and accordingly generating the output stereo image pair to the stereo display apparatus 101 for video/image playback.”)
8. Claim 12 is similar in scope to claim 1 except for additional limitations that Cheng discloses: An electronic device, comprising:
at least one processor; and
a storage apparatus configured to store at least one program, wherein the at least one program, when executed by the at least one processor, causes the at least one processor to implement a binocular image generation method, wherein the binocular image generation method comprises: (Cheng, [0008], “According to a third aspect of the present invention, an exemplary non-transitory machine-readable medium which stores a program code is disclosed. When the program code is executed by a processor, the processor is instructed to perform following steps: performing a disparity analysis upon an input stereo image pair, ... and accordingly generating an output stereo image pair for playback.” and [0038], “In the aforementioned embodiment, the auto-convergence system 100 may be implemented using pure hardware.”)
9. Claim 13, which is similar in scope to claims 1 and 12, is thus rejected under the same rationale as described above.
10. Claim 14, which is similar in scope to dependent claim 2 and independent claim 12, is thus rejected under the same rationale as described above.
11. Claim 15, which is similar in scope to dependent claims 2 and 3 and independent claim 12, is thus rejected under the same rationale as described above.
12. Claim 21, which is similar in scope to dependent claim 9 and independent claim 12, is thus rejected under the same rationale as described above.
Claim Rejections - 35 USC § 103
13. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
14. Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng (US-2014/0254917-A1) in view of Ha et al. (US-2007/0081716-A1, hereinafter "Ha").
15. As per claim 4, Cheng discloses: The method according to claim 1, wherein the generating the binocular image based on the target depth value and depth values of a plurality of pixels in the original image comprises:
determining, based on the target depth value and the depth values of the plurality of pixels in the original image, (See rejection for claim 1.)
16. Cheng doesn't explicitly disclose but Ha discloses: [[determining, based on the target depth value and the depth values of the plurality of pixels in the original image,]] a plurality of displacement vectors of the plurality of pixels in the original image in each of a left-eye image and a right-eye image that are to be generated; and (Ha, [0045]-[0046], “FIG. 7 illustrates block-based disparity estimation (DE) according to an embodiment of the present invention. Referring to FIG. 7, a left-eye image is divided into N×N blocks of equal size. Blocks of a right-eye image which are most similar to corresponding blocks in the left-eye image are estimated using a sum of absolute difference (SAD) or a mean of absolute difference (MAD). In this case, a distance between a reference block and an estimated block is defined as a disparity vector (DV). Generally, a DV is assigned to each pixel in the reference image. However, to reduce the amount of computation required, it is assumed that the DVs of all pixels in a block are approximately the same in the block-based DE. The performing of DE on each pixel to obtain the DV for each pixel is called pixel-based DE.”)
processing the plurality of pixels in the original image based on the plurality of displacement vectors, to generate the left-eye image and the right-eye image. (Ha, [0068], “The left-eye image and the right-eye image are horizontally moved based on the determined horizontal movement value and the disparities between the left-eye image and the right-eye image are adjusted (S126). The disparity-adjusted left- and right-eye images are output and displayed.” and [0046]-[0047], “The performing of DE on each pixel to obtain the DV for each pixel is called pixel-based DE. The block-based DE or the pixel-based DE is used to estimate a disparity.” and [0062]-[0063], “The histogram generation unit 13 estimates the disparities between the right-eye image and the left-eye image, measures the frequency with which the estimated disparities occur, and generates a histogram for the disparities and the frequency. In this case, the block-based DE or the pixel-based DE described above or other methods may be used. The horizontal movement value determination unit 15 receives the generated histogram from the histogram generation unit 13 and determines a horizontal movement value for the left- and right-eye images.”)
17. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Cheng to include the disclosure of determining displacement vectors between pixels of the left and right eye images and processing the pixels based on the vectors to generate a stereo image pair of Ha. The motivation for this modification could have been to allow a user to fully adjust the image disparity and the zero-disparity plane. Some users of a stereo viewing system may have higher or lower tolerance for image disparity. This would allow a user to customize how they view stereo content so they are comfortable while viewing.
18. Claim 16, which is similar in scope to dependent claim 4 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 4.
19. Claims 5-6 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng (US-2014/0254917-A1) in view of Rowell et al. (US-2019/0158813-A1, hereinafter "Rowell").
20. As per claim 5, Cheng discloses: The method according to claim 1, (See rejection for claim 1.)
21. Cheng doesn't explicitly disclose but Rowell discloses: wherein before the generating the binocular image based on the target depth value and depth values of a plurality of pixels in the original image, the method further comprises: (Rowell, [0166], “Image data included in compressed image sections 1605 (“Lc” and “Rc”) generated by the image data preprocessor 1703 is written to memory readable by other components of the auto re-calibration subsystem 1702. Image data included in compressed image sections 1605 or uncompressed image frames is then filtered by the filtering module 1705 to increase accuracy of the image data evaluated with disparity analysis.”)
filtering out a pixel whose depth value is less than a first threshold and a pixel whose depth value is greater than a second threshold from the original image, wherein the first threshold is less than the second threshold. (Rowell, [0187], “A depth filtering function is a third filtering function that may be implemented in the filtering module 1705. The depth filtering function excludes image data from image sections incorporating objects positioned a certain distance away from the stereo camera device. In one example, the depth filtering function rejects image data included in image sections containing close objects because the extreme horizontal and/or vertical shifts applied to image sections including close objects elsewhere in projection process interfere with re-calibration.” and [0189], “One non-limiting example depth filtering function determines a depth metric for each image section then filters image data in the image sections by comparing the depth metrics to a depth filtering threshold. The depth metric describes the distance between the stereo camera device and the objects included in an image section. Depth maps, point clouds, 3D scans, and distance measurements may all be used to generate depth metrics. One non-limiting example distance measurement is equivalent to 1/the horizontal shift (in pixels) applied to images captured by the stereo camera module. Depth filtering thresholds for evaluating depth metrics include 20 cm for camera modules having small zoom ranges and short focal lengths and 1 m for camera modules having moderate to large zoom ranges and average to long focal lengths.” and [0190], “In filtering routines including three filtering layers, image data may be required to pass three filtering thresholds to be incorporated into a disparity analysis. Alternatively, image data may only need to pass a majority or at least one of the three filtering thresholds (e.g., 2 of 3 or 1 of 3). In embodiments where image data is required to meet or exceed a subset of the filtering thresholds, the required filtering thresholds may be the same or different (e.g., image sections must pass both the depth filtering function and the standard deviation filtering function; image sections must pass the correlation filtering function and at least one other filtering function; or image sections must pass and any two filtering functions).”; Examiner’s note: The filtering routines disclosed by Rowell would have the capability to filter out pixels for either being less than and/or greater certain defined thresholds.)
22. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Cheng to include the disclosure of filtering out pixels whose depth value is either less than or greater than defined thresholds from the image of Rowell. The motivation for this modification could have been to assist in the visual removal of objects that are either too close or too far away in the stereo images. In doing so, this would help prevent awkward distractions and place more emphasis on the objects closer to the zero-disparity plane where a user is more likely to be focused.
23. As per claim 6, Cheng in view of Rowell discloses: The method according to claim 1, wherein before the generating the binocular image based on the target depth value and depth values of a plurality of pixels in the original image, the method further comprises: (Rowell, [0166], “Image data included in compressed image sections 1605 (“Lc” and “Rc”) generated by the image data preprocessor 1703 is written to memory readable by other components of the auto re-calibration subsystem 1702. Image data included in compressed image sections 1605 or uncompressed image frames is then filtered by the filtering module 1705 to increase accuracy of the image data evaluated with disparity analysis.”)
performing filtering on the depth values of the plurality of pixels in the original image. (Rowell, [0187], “A depth filtering function is a third filtering function that may be implemented in the filtering module 1705. The depth filtering function excludes image data from image sections incorporating objects positioned a certain distance away from the stereo camera device. In one example, the depth filtering function rejects image data included in image sections containing close objects because the extreme horizontal and/or vertical shifts applied to image sections including close objects elsewhere in projection process interfere with re-calibration.” and [0189], “One non-limiting example depth filtering function determines a depth metric for each image section then filters image data in the image sections by comparing the depth metrics to a depth filtering threshold.”)
24. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Cheng to include the disclosure of performing filtering on the depth values of the plurality of pixels in the original image of Rowell. The motivation for this modification could have been to help smooth out any potential image artifacts in the stereo imagery. In addition, it could also help to make sure any drastic changes in image disparity are dampened so that viewing the stereo imagery is more comfortable to the user.
25. Claim 17, which is similar in scope to dependent claim 5 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 5.
26. Claim 18, which is similar in scope to dependent claim 6 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 6.
27. Claims 7-8 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng (US-2014/0254917-A1) in view of Liao et al. (US-2012/0113093-A1, hereinafter "Liao").
28. As per claim 7, Cheng discloses: The method according to claim 1, wherein after the generating the binocular image, the method further comprises: (See rejection for claim 1.)
29. Cheng doesn't explicitly disclose but Liao discloses: determining a void region in the binocular image; and (Liao, [0034], “To reliably fill the holes and produce a convincing, realistic result, the system should specifically target the image inpainting for a stereo setting, taking into account the available depth (disparity) information, the local geometric saliency, as well as intensity information.” and [0036], “Referring to FIG. 5, with different techniques more suitable for different types of content (low frequency and high frequency), where natural images typically exhibit both types of content, a hybrid framework facilitates the inpainting for both cases. The first pass fills holes in low-frequency image regions using a first filter, such as an adaptive linear interpolation, and afterwards the second pass processes high-frequency image regions using a second filter, such as a non-linear exemplar-based inpainting.” and [0037], “During the first pass for low frequency inpainting, the holes in a low-frequency image region in IS are typically surrounded by pixels that are similar in intensity values as well as in disparity values. On a single scanline, a hole is represented as a succession of empty pixels of length γ.” and [0039], “The second pass of inpainting addresses high-frequency content in the image, filling pixels that are left unprocessed (left empty) by the first pass in IS′, DS′, and ES′. In order to preserve high-frequency details, the technique may employ a non-linear exemplar-based approach to synthesize the remaining holes. Since an exemplar-based inpainting relies both on intensity similarity and disparity similarity in evaluating a matching cost, the technique may make an explicit assumption regarding occlusion. For example, the technique may assume all occlusions in the stereo image pair to be two-layer occlusions.”)
performing image gradient diffusion from an edge with a large depth value in the void region to an edge with a small depth value, to fill the void region. (Liao, [0023], “Since the technique performs disparity scaling, rather than constant offsetting, holes 320 are created in the synthesized view IS ... The view optimization 400 may employ a one or a two-pass, depth-assisted inpainting approach 410 to optimize the synthesis by filling in the holes, producing a complete 420 (i.e. no-holes) new view IP and a complete reference disparity map DP.” and [0034], “To reliably fill the holes and produce a convincing, realistic result, the system should specifically target the image inpainting for a stereo setting, taking into account the available depth (disparity) information, the local geometric saliency, as well as intensity information.” and [0021], “To summarize, based on the original stereo image pair, the technique preferably generates a “continuous range” (e.g., three or more) of interpolation as well as extrapolation of new view points with adjusted depth, according to a viewer's preference. By employing scaling of disparity values, rather than constant offsetting, this preserves the spatial variation of disparity (inverse depth) across the image, which in turn preserves the underlying scene geometry (depth). Also, by synthesizing a new view using local image intensity values, disparity values, and geometric saliency information, as opposed to relying only on intensity further refines the synthesis process by separately processing the low and high frequency content in the stereo pair. Further, the reduction of grid quantization artifacts in a local manner also preserves details and reduces blurring the synthesized virtual view.” and [0035], “The inpainting technique may be linear or non-linear. Such linear techniques may include, for example, scanline interpolation and extrapolation. Such linear techniques tend to be computationally efficient, and tend to be robust when intensity variations are low (low frequency) in an image region requiring inpainting. Such non-linear techniques tend to be more computationally complex, and may include for example, partial differential equations or utilize exemplars in filling holes.”)
30. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Cheng to include the disclosure of determining a void region in the binocular image and performing image gradient diffusion to fill the void region of Liao. The motivation for this modification could have been to prevent annoying or distracting “gaps” (or void) within the stereo images. In a stereo application, these “gaps” may make a user uncomfortable or even nauseous, especially at the edges of objects of different disparities. By filling these holes, it helps preserve the stereo experience, removing the distraction, and making the watching experience more pleasant.
31. As per claim 8, Cheng in view of Liao discloses: The method according to claim 7, wherein after the void region is filled, the method further comprises:
performing filtering on the filled void region. (Liao, [0019]-[0020], “To provide in-painting of the holes in the synthesized image created as a result of scaled offsets should, in addition to using local intensity values, model the local disparity values and local geometric saliency. ... The technique may further eliminate grid quantization artifacts that are a result of the inherently discrete, or quantized, nature of disparity values, where a given pixel is only allowed to assume an integer lateral offset number. This may be done by adaptively filtering the synthesized virtual view in a small neighborhood where such artifacts are detected. By restricting the filtering to a small neighborhood, the technique reduces the artifacts while preserving the details in the synthesized view, without introducing excessive undesirable blurring.” and [0052], “With knowledge of the locations of artifacts, the technique proceeds to filter out the artifacts. ... The operation effectively low-pass filters a very small neighborhood, typically 4×1 or 3×1, centered on and overwriting the artifact pixel. By restricting the filtering to a small neighborhood, the system is able to eliminate the artifacts while preserving the details in the synthesized view, without introducing undesired blurring.” and Claim 14. “The method of claim 7 further comprising a filter to reduce grid quantization artifacts.” and Claim 15, “The method of claim 14 wherein adaptive filtering is used to reduce grid quantization artifacts.”)
32. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 7 of Cheng to include the disclosure of performing filtering on the filled void region of Liao. The motivation for this modification could have been to help smooth out any strange image artifacts the image filling process created when removing the “gaps” (or void). This process would help making the filled regions more seamless with the stereo images and much less likely to be noticed or distracting.
33. Claim 19, which is similar in scope to dependent claim 7 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 7.
34. Claim 20, which is similar in scope to dependent claims 7 and 8 and independent claim 12, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 8.
35. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng (US-2014/0254917-A1) in view of Freeman et al. (US-2018/0160098-A1, hereinafter "Freeman").
36. As per claim 10, Cheng discloses: The method according to claim 9, (See rejection for claim 9.)
37. Cheng doesn't explicitly disclose but Freeman discloses: wherein the video frame comprises a video frame in a live stream, and the salient region comprises a facial region of a live streamer. (Freeman, Abstract, “A system, method and software for producing 3D effects in a video of a physical scene. The 3D effects can be observed when the video is viewed, either during a live stream or later when viewing the recorded video. A reference plane is defined. The reference plane has peripheral boundaries. A live event is viewed with stereoscopic video cameras.” and [0037], “Depending upon the 3D effect being created, the stereoscopic cameras 20L, 20R are oriented so that their focal points are on the reference plane 46 and/or their lines of sight intersect at the reference plane 46. That is, the two stereoscopic cameras 20L, 20R achieve zero parallax at the reference plane 46.” and [0053], “The video producer and/or 3D effects technician also selects a reference plane 46 within the physical scene. See Block 82. The video producer and/or 3D effects technician then selects objects in the view of the stereoscopic cameras 20L, 20R that will be identified in production as primary subjects 42, secondary subjects 60 and background subjects 62. See Block 84. Using the reference plane 46 and the selected subjects, the video producer and/or 3D effects technician can determine the boundaries for the video scene 40 being produced. See Block 86.” and [0057], “Referring to FIG. 14, a studio setting 102 is shown. The studio setting 102 has one of more sets of stereoscopic cameras 104 positioned to image a person or other real object within a known defined area. In the shown example, the person is a teacher 106 standing in front of a blackboard 108. In this scenario, the blackboard 108 can be selected as the reference plane.”; Examiner’s note: As disclosed in ¶ [0053], a video producer can select a reference plane within a physical scene. For the example of the teacher and the blackboard described in ¶ [0057], the producer has the ability to choose the reference plane. Instead of the blackboard, the producer can rather choose the teacher’s face as a reference plane.)
38. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 9 of Cheng to include the disclosure of a live stream of stereo video frames wherein the salient region comprises of a facial region of a live streamer of Freeman. The motivation for this modification could have been to adapt the zero-disparity plane for a live stream where the plane is focused on a live streamer. In a stereo live stream application, a user would likely be focused on a live streamer. By adjusting the zero-disparity plane to the facial region of the streamer, the user may find watching the stream more comfortable and easier to focus on.
Conclusion
39. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614