DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Amendment
This office action is responsive to the amendment received 09/16/2026.
In the response to the Non-Final Office Action 08/17/2026, the applicant states that claims 1, 10, and 16-20 are currently amended. Claims 1-20 are pending in the present application.
Claims 1, 10, and 16-20 have been amended. In summary, claims 1-20 are pending in current applications.
Response to Arguments
Applicant's arguments filed 09/16/2026 have been fully considered.
Regarding 35 U.S.C 112 (a) rejection, the amendment has cured the basis of 35 U.S.C 112 (a) rejection. Therefore, the 35 U.S.C 112 (a) rejection is hereby withdrawn.
Regarding to claim 1, the applicant argues that cited arts fail to teach or suggest "wherein the convex polygon analysis method defines a convex polygon comprising the second object based on coordinate information of the second object and determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon." The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Creusot discloses "wherein the convex polygon analysis method defines a convex polygon comprising the second object based on coordinate information of the second object” For example, in paragraph [0009], Creusot teaches defining a similar polygon around such objects within the region of interest. In paragraph [0030], Creusot teaches projecting the three-dimensional point cloud with coordinates into a two-dimensional range image; Creusot further teaches defining an occlusion polygon around foreground objects, i.e. second project. In paragraph [0033], Creusot teaches converting the three-dimensional sensor signal into a two-dimensional range image/sparse depth map. In paragraph [0035], Creusot teaches the occlusion polygon is defined around a foreground object in the range image that is in proximity to the focal object; Creusot further teaches the occlusion polygon is typically formed as a convex polygon; Creusot furthermore teaches the range image is cropped to the dimensions of the region of interest.
Okada discloses “determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon”. For example, in Fig. 1 and paragraph [0036], Okada teaches the three-dimensional figure data A includes data representing the positions of vertexes of polygons obtained by approximating the human body surface by a set of polygons, and data representing the positions of joints of the human body;
PNG
media_image1.png
528
562
media_image1.png
Greyscale
. In paragraph [0040], Okada teaches the clothes figure data C includes the three-dimensional shape and the surface patterns of the clothing. In Fig. 4 and paragraph [0071], Okada teaches the clothes simulation unit 4 expresses the surfaces of the clothes by a set of polygons based on the clothes figure data C. In paragraph [0072], Okada teaches the positions of vertexes of the calculated polygons. In Fig. 4 and paragraph [0073], Okada teaches three-dimensional coordinates of vertexes of polygons on the clothes, i.e. ranges, include a horizontal coordinate range, i.e. x and y coordinates, and a vertical coordinate range, i.e. z coordinate, of vertexes of a polygon; Okada further teaches the display unit 5 displays the images of the clothes overlapped on the images of the person obtained from the image capturing unit 1; Okada furthermore teaches the display portion 5 finds and determines an occlusion relationship between the clothes and the person within the image plane. In addition, Okada suggests this occlusion relationship is based upon the three-dimensional coordinates of vertexes of polygons on the clothes computed by the clothes simulation unit 4 and based upon the three-dimensional coordinates of points of the three-dimensional figure models; Okada further suggests the display unit 5 synthesizes the images of the clothes on the images of the person based on the occlusion relationship and displays the synthesized images.
Goddard discloses “both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon”. For example, in Fig. 2 and col. 2, lines 10-25, Goddard teaches coordinate data at polygon corners X, Y, Z; Goddard further teaches polygon ranges in vertical direction and horizontal direction as illustrated in Fig. 2;
PNG
media_image2.png
374
750
media_image2.png
Greyscale
. Goddard furthermore teaches multiple polygons of tree and building as illustrated in Fig. 2.
Regarding to claim 1, the applicant argues that A prima facie case of obviousness has therefore not been established for claims 1, 10, and 16. The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
In response to applicant's argument that a prima facie case of obviousness has therefore not been established for claims 1, 10, and 16, the examiner recognizes that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the motivation for combining Zhou and Goddard would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65. The motivation for combing Zhou, Goddard, and Creusot would have been to detect occlusions; to identify occlusions by defining an occlusion polygon around the foreground objects based upon the distance information incorporated in the range image; to detect an occlusion via the occlusion polygon 410 as taught by Creusot in paragraphs [0033], [0035], and [0043]. The motivation for combining Zhou, Goddard, Creusot, and Okada would have been to find an occlusion relationship between the clothes and the person within the image plane; to synthesize the images of the clothes on the images of the person based on the occlusion relationship and displays the synthesized images as taught by Okada in paragraphs [0071-0073]. “the law does not require that the references be combined for the reasons contemplated by the inventor” {In re Beattie, 974 F.2d 1309, 1314 (Fed. Cir. 1992)(prior citations omitted)). It is concluded that the Examiner’s articulated reasoning provides a rational underpinning to support the legal conclusion of obviousness. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398,418 (2007).
Regarding to claims 2, 11, and 17, the applicant argues that Zhou therefore does not perform segmentation on an image that comprises the second object but not the first object. The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Zhou discloses “perform segmentation on an image that comprises the second object but not the first object”. For example, in Fig. 2 and paragraph [0054], Zhou teaches categorize each pixel of the selected frame or image as a foreground pixel or a background pixel. In paragraph [0055], Zhou teaches if a pixel is determined to be a background pixel, a mask value of 0 is assigned to the pixel, and if a pixel is determined to be a foreground pixel, a mask value of 255 may be assigned to the pixel; determine a mask area with value 0; Zhou further teaches the area of the second object is a mask area with value 0. In Fig. 5 and paragraphs [0124-0125], Zhou teaches foreground pixels of the mask are in white color while background pixels are in black color. In Fig. 8 and paragraph [0131], Zhou teaches a foreground is generated by segmenting the input video according to some implementations; Zhou further teaches the foreground is separated from the background of the conference room, which has been replaced with a mountain scene; Zhou furthermore teaches the segmentation is accurate in the presence of motion and the corresponding output frame correctly segments the foreground.
Regarding to claims 6, 7, and 8, the applicant argues that cited arts do not teach or suggest claims 6, 7, and 8. The arguments have been fully considered, but they are not persuasive.
Claims 2-20 are not allowable due to the similar reasons as discussed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, and 9-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou (US 20210019892 A1) in view of Goddard (US 6104402 A), in view of Creusot (US 20200081448 A1), and further in view of Okada (US 20060202986 A1).
Regarding to claim 1 (Currently Amended), Zhou discloses an image synthesizing method ([0031]: the methods, devices, and computer-readable media with instructions obtain a foreground video; generate a composite video that includes a foreground video segmented from a captured scene, overlaid on a background different from the original captured background; provide the foreground video in a video conference; Fig. 1; [0032]: server device 104 provides a video application 152b, e.g., a video calling application, an augmented reality application, and a virtual reality application; Fig. 2; [0095]; Fig. 8; [0131]: in each of the output frames, the foreground is separated from the background of the conference room, which has been replaced with a mountain scene; [0134]: processor; GPU; ASIC; [0135]: CPU and GPU perform computation in parallel; the GPU or parallel processor include a GPU memory separate from main memory 904), wherein the method comprises:
determining a posture of a first object in an image and a relative position relationship between the first object and a second object in the image ([0031]: segment and determine a foreground video from a captured scene; [0047]: each image frame includes color data and depth data, i.e. relative position, for a plurality of pixels; [0051]: capture and determine depth data at 15 frames per second; depth data is relative position; Fig. 2; [0054]: determine whether the depth value is within the depth range; categorize each pixel of the selected frame or image as a foreground pixel or a background pixel; Fig. 2; [0058]: detect a head bounding box; the head bounding box may specify a region of the frame that includes a head or is likely to include a head; [0059]: the trimap classifies each pixel of the image as being one of known foreground, known background, and unknown; Fig. 5; [0124]: the foreground of the video frame (502) includes a person with raised arms, i.e. posture of a person;
PNG
media_image3.png
310
482
media_image3.png
Greyscale
; the background of the video frame includes an office environment with a whiteboard and another person at a workstation with their back to the person in the foreground; foreground and background are relative position; Fig. 5; [0125]: a segmentation mask categorizes each pixel of the frame (502) as foreground pixel or background pixel;
PNG
media_image4.png
392
464
media_image4.png
Greyscale
; determine that a person raises his hands and arms and the person is in foreground; foreground pixels of the mask are in white color while background pixels are in black color; Fig. 8; [0131]: the postures include raising a hand with fingers separated in frame 822; and turning the hand in frame 832.);
determining, based on the posture and the relative position relationship, a foreground component comprised in a virtual scene ([0031]: segment and determine a foreground video from a captured scene; [0054]: categorize each pixel of the selected frame or image as a foreground pixel or a background pixel based on depth data; [0059]: the trimap classifies each pixel of the image as being one of known foreground, known background, and unknown; [0083]: create and determine background and foreground color models globally; Fig. 3; [0101]: the segmentation mask is an initial segmentation mask, and is determined from depth data of a video frame; Fig. 5; [0125]: a segmentation mask categorizes each pixel of the frame (502) as foreground pixel or background pixel;
PNG
media_image3.png
310
482
media_image3.png
Greyscale
PNG
media_image4.png
392
464
media_image4.png
Greyscale
; determine that a person raises his hands and the person is a foreground in a scene as illustrated in Fig. 5; foreground pixels of the mask are in white color while background pixels are in black color.); and
synthesizing an image of the virtual scene based on the foreground component, an image part comprising the first object, and the relative position relationship ([0031]: generate and synthesize a composite video that includes a foreground video segmented from a captured scene, overlaid on a background different from the original captured background; Fig. 2; [0091]: render the foreground video; generate a plurality of frames that include the foreground, segmented using the binary mask; display the foreground video in a video calling application or other application; [0092]: the user indicates a preference to substitute the background with a particular scene, and such background is synthesized and displayed with the foreground video; Fig. 8; [0131]: an output video includes a foreground generated by segmenting the input video;
PNG
media_image5.png
566
542
media_image5.png
Greyscale
; an output video is a synthesized image; in each of the output frames, the foreground is separated from the background of the conference room, which has been replaced with a mountain scene).
Zhou fails to explicitly disclose:
wherein the relative position relationship comprises a spatial relationship or an occlusion relationship, wherein the occlusion relationship is based on a convex polygon analysis method, wherein the convex polygon analysis method defines a convex polygon comprising the second object based on coordinate information of the second object and determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon;
In same field of endeavor, Goddard teaches:
wherein the relative position relationship comprises a spatial relationship or an occlusion relationship (or is optional; Fig. 1, col. 1, lines 40-55: when rendering a person as a moving object 1 and depicting a building 2 and a tree 3 as background objects, the
moving object 1 is represented in preference to the back ground objects; a person is in front of the building and tree as illustrated in Fig. 1;
PNG
media_image6.png
470
786
media_image6.png
Greyscale
; Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
; a person is occluded by a tree when a person is at positions c and d as illustrated in Fig. 2; Fig. 10; col. 14, lines 15-25: display in combination of the background object and the moving object at a step 60 shown in FIG. 10; synthesizing process; col. 16, lines 25-37: the depth position of the moving object 1 is farther than that of the background scene 5F and accordingly invisibly hidden by any background object in the background scene; col. 17, lines 36-45: when the moving object 1 comes to a position (c) beside the tree 3, it passes on a deeper side of the tree 3 so that its portion hidden by leaves of the tree 3; col. 17, lined 50-60: the lower half portion of the moving object 1, upon being displayed, is invisibly hidden by the trunk.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou to include wherein the relative position relationship comprises a spatial relationship or an occlusion relationship as taught by Goddard. The motivation for doing so would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65.
Zhou in view of Goddard fails to explicitly disclose:
wherein the occlusion relationship is based on a convex polygon analysis method, wherein the convex polygon analysis method defines a convex polygon comprising the second object based on coordinate information of the second object and determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon.
In same field of endeavor, Creusot teaches:
wherein the occlusion relationship is based on a convex polygon analysis method ([0009]: an occlusion polygon is defined around foreground objects in the region of interest to identify occlusions to traffic lights; the occlusion polygon and related background polygon are typically configured as a convex polygon; [0035]: the occlusion polygon module 206 identifies foreground objects/occlusions by defining an occlusion polygon around the foreground objects based upon the distance information incorporated in the range image; the occlusion polygon is typically formed as a convex polygon; [0043]: detect an occlusion via the occlusion polygon 410), wherein the convex polygon analysis method defines a convex polygon comprising the second object based on coordinate information of the second object ([0009]: define a similar polygon around such objects within the region of interest; [0030]: project the three-dimensional point cloud with coordinates into a two-dimensional range image with coordinates; define an occlusion polygon around foreground objects, i.e. second project; [0035]: the occlusion polygon is defined around a foreground object in the range image that is in proximity to the focal object; the occlusion polygon is typically formed as a convex polygon; the range image is cropped to the dimensions of the region of interest).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou in view of Goddard to include wherein the occlusion relationship is based on a convex polygon analysis method, wherein the convex polygon analysis method defines a convex polygon surrounding the second object based on coordinate information of the second object as taught by Creusot. The motivation for doing so would have been to detect occlusions; to identify occlusions by defining an occlusion polygon around the foreground objects based upon the distance information incorporated in the range image; to detect an occlusion via the occlusion polygon 410 as taught by Creusot in paragraphs [0033], [0035], and [0043].
Zhou in view of Goddard and Creusot fails to explicitly disclose:
determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon.
In same field of endeavor, Okada teaches
determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon (Fig. 1; [0036]: the three-dimensional figure data A includes data representing the positions of vertexes of polygons obtained by approximating the human body surface by a set of polygons, and data representing the positions of joints of the human body;
PNG
media_image1.png
528
562
media_image1.png
Greyscale
; [0040]: the clothes figure data C includes the three-dimensional shape and the surface patterns of the clothing; Fig. 4; [0071]: the clothes simulation unit 4 expresses the surfaces of the clothes by a set of polygons based on the clothes figure data C.; [0072]: the positions of vertexes of the calculated polygons; Fig. 4; [0073]: three-dimensional coordinates of vertexes of polygons on the clothes, i.e. ranges, include a horizontal coordinate range, i.e. x and y coordinates, and a vertical coordinate range, i.e. z coordinate, of vertexes of a polygon; the display unit 5 displays the images of the clothes overlapped on the images of the person obtained from the image capturing unit 1; the display portion 5 finds and determines an occlusion relationship between the clothes and the person within the image plane; this occlusion relationship is based upon the three-dimensional coordinates of vertexes of polygons on the clothes computed by the clothes simulation unit 4 and based upon the three-dimensional coordinates of points of the three-dimensional figure models; the display unit 5 synthesizes the images of the clothes on the images of the person based on the occlusion relationship and displays the synthesized images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou in view of Goddard and Creusot to include “determines the occlusion relationship based on a determination that both a horizontal coordinate range and a vertical coordinate range of vertexes of a polygon comprising the first object intersect a horizontal coordinate range and a vertical coordinate range of vertexes of the convex polygon” as taught by Okada. The motivation for doing so would have been to find an occlusion relationship between the clothes and the person within the image plane; to synthesize the images of the clothes on the images of the person based on the occlusion relationship and displays the synthesized images as taught by Okada in paragraphs [0071-0073].
Regarding to claim 2 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 1, wherein the method further comprises:
determining an area of the second object in the image by performing semantic segmentation on an image that comprises the second object but does not comprise the first object (Zhou; Fig. 2; [0054]: categorize each pixel of the selected frame or image as a foreground pixel or a background pixel; [0055]: if a pixel is determined to be a background pixel, a mask value of 0 is assigned to the pixel, and if a pixel is determined to be a foreground pixel, a mask value of 255 may be assigned to the pixel; determine a mask area with value 0; the area of the second object is a mask area with value 0; Fig. 5; [0124-0125]: foreground pixels of the mask are in white color while background pixels are in black color; Fig. 8; [0131]).
Regarding to claim 3 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 2, wherein the determining a relative position relationship between the first object and a second object (same as rejected in claim 1) comprises:
determining the relative position relationship between the first object and the second object based on coordinate information of an area of the first object in the image and coordinate information of the area of the second object in the image (Zhou; [0047]: each image frame includes color data and depth data, i.e. relative position, for a plurality of pixels; [0051]: capture and determine depth data at 15 frames per second; depth data is relative position; [0054]: the depth value for each pixel is compared to the depth range to determine whether the depth value is within the depth range; if the depth value is within the depth range, the pixel is categorized as a foreground pixel; if the depth value is outside the depth range, the pixel is categorized as a background pixel; [0060]: the mask boundary corresponds to locations, e.g., expressed in pixel coordinates, where at least one foreground pixel is adjacent to at least one background pixel in the initial segmentation mask; Fig. 5; [0124-0125]: the left side of the image includes background as illustrated in in Fig. 5;
PNG
media_image8.png
296
384
media_image8.png
Greyscale
; Fig. 8; [0131]).
Regarding to claim 4 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 1, wherein the determining a foreground component comprised in a virtual scene (same as rejected in claim 1) comprises:
selecting, based on a relative size of the first object and the second object in the image, the foreground component whose size matches the first object (Zhou; [0052]: the size of the down-sampled video is selected based on the size of the received video; [0090]: the respective binary mask is utilized to obtain a foreground video; [0111]: a location and a size are determined to detect the location of the head area; Fig. 5; [0124-0125]: the size of mask matches the size of a person as illustrated in Fig. 5; foreground pixels of the mask are in white color while background pixels are in black color.).
Regarding to claim 9 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 1, wherein the first object is a person or a virtual image, and the second object is a static object (Zhou; Fig. 5; [0124-0125]: the foreground of the video frame (502) includes a person with raised arms; the background of the video frame includes an office environment with a whiteboard in the foreground; an office environment with a whiteboard is a static object; Fig. 8; [0131]: an output video is a synthesized image; in each of the output frames, the foreground is separated from the background of the conference room, which has been replaced with a mountain scene; conference room is a static object).
Regarding to claim 10 (Currently Amended), Zhou discloses an image synthesizing apparatus, wherein the apparatus ([0031]: the methods, devices, and computer-readable media with instructions obtain a foreground video; generate a composite video that includes a foreground video segmented from a captured scene, overlaid on a background different from the original captured background; provide the foreground video in a video conference; Fig. 1; [0032]: server device 104 provides a video application 152b, e.g., a video calling application, an augmented reality application, and a virtual reality application; Fig. 2; [0095]; Fig. 8; [0131]: in each of the output frames, the foreground is separated from the background of the conference room, which has been replaced with a mountain scene; [0134]: processor; GPU; ASIC; [0135]: CPU and GPU perform computation in parallel; the GPU or parallel processor include a GPU memory separate from main memory 904) comprises:
at least one processor configured to ([0134]: processor; GPU; ASIC):
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 10.
Regarding to claim 11 (Previously Presented), Zhou in view of Goddard, Creusot, and Okada discloses the apparatus according to claim 10, wherein the at least one processor is further (same as rejected in claim 10):
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 11.
Regarding to claim 12 (Previously Presented), Zhou in view of Goddard, Creusot, and Okada discloses the apparatus according to claim 10, wherein the at least one processor is further configured to (same as rejected in claim 10):
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 12.
Regarding to claim 13 (Previously Presented), Zhou in view of Goddard, Creusot, and Okada discloses the apparatus according to claim 10, wherein the at least one processor is further configured to (same as rejected in claim 10):
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 13.
Regarding to claim 14 (Presented Previously), Zhou in view of Goddard, Creusot, and Okada discloses the apparatus according to claim 10, and the at least one processor is further configured to (same as rejected in claim 10):
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 14.
Regarding to claim 15 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the apparatus according to claim 10,
The rest claim limitations are similar to claim limitations recited in claim 9. Therefore, same rational used to reject claim 9 is also used to reject claim 15.
Regarding to claim 16 (Currently Amended), Zhou discloses a non-transitory computer-readable storage media storing computer instructions that configure at least one processor, upon execution of the instructions, to perform the following steps ([0031]: the methods, devices, and computer-readable media with instructions obtain a foreground video; generate a composite video that includes a foreground video segmented from a captured scene, overlaid on a background different from the original captured background; provide the foreground video in a video conference; Fig. 1; [0032]: server device 104 provides a video application 152b, e.g., a video calling application, an augmented reality application, and a virtual reality application; Fig. 2; [0095]; Fig. 8; [0131]: in each of the output frames, the foreground is separated from the background of the conference room, which has been replaced with a mountain scene; [0134]: processor; GPU; ASIC; [0135]: CPU and GPU perform computation in parallel; the GPU or parallel processor include a GPU memory separate from main memory 904; [0138]: memory 904 is typically provided in device 900 for access by the processor 902; [0140]: any of software in memory 904 is alternatively be stored on any other suitable storage location or computer-readable medium):
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 16.
Regarding to claim 17 (Currently Amended), Zhou in view of Goddard, Creusot, and Okada discloses the non-transitory computer-readable storage medium according to claim 16, wherein the steps further comprises:
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 17.
Regarding to claim 18 (Currently Amended), Zhou in view of Goddard, Creusot, and Okada discloses the non-transitory computer-readable storage medium according to claim 16,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 18.
Regarding to claim 19 (Currently Amended), Zhou in view of Goddard, Creusot, and Okada discloses the non-transitory computer-readable storage medium according to claim 16,
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 19.
Regarding to claim 20 (Currently Amended), Zhou in view of Goddard, Creusot, and Okada discloses the non-transitory computer-readable storage medium according to claim 16,
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 20.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou (US 20210019892 A1) in view of Goddard (US 6104402 A), in view of Creusot (US 20200081448 A1), and further in view of Okada (US 20060202986 A1).
Regarding to claim 5 (Previously Presented), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 1, wherein the relative position relationship comprises the synthesizing an image of the virtual scene comprises: synthesizing the image of the virtual scene (Zhou; [0091]: render the foreground video; Fig. 5; [0124-0125]; Fig. 8; [0131]).
Zhou fails to explicitly disclose: the occlusion relationship between the first object and the second object; wherein the first object and the foreground component satisfy the occlusion relationship in the synthesized image of the virtual scene.
In same field of endeavor, Goddard teaches:
the occlusion relationship between the first object and the second object (Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
; Fig. 10; col. 14, lines 15-25: display in combination of the background object and the moving object at a step 60 shown in FIG. 10; synthesizing process; col. 16, lines 25-37: the depth position of the moving object 1 is farther than that of the background scene 5F and accordingly invisibly hidden by any background object in the background scene; col. 17, lines 36-45: when the moving object 1 comes to a position (c) beside the tree 3, it passes on a deeper side of the tree 3 so that its portion hidden by leaves of the tree 3; col. 17, lined 50-60: the lower half portion of the moving object 1, upon being displayed, is invisibly hidden by the trunk.);
wherein the first object and the foreground component satisfy the occlusion relationship in the synthesized image of the virtual scene (Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
; col. 17, lines 55-65: display an image to pass the moving object 1 through the front of the building 2 and the tree 3; col. 18; lines 10-25: when the color data stores by the image buffer 152 is read out in synchronism with raster Scanning over the display 31, an image as shown in FIG. 2 is displayed.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou to include the occlusion relationship between the first object and the second object; wherein the first object and the foreground component satisfy the occlusion relationship in the synthesized image of the virtual scene as taught by Goddard. The motivation for doing so would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65.
Regarding to claim 6 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 5, wherein
updating the foreground component comprised in the virtual scene (Zhou; Fig. 8; [0131]: raise a hand with fingers separated in frame 822; and turns the hand in frame 832.), and
synthesizing the image of the virtual scene based on an updated foreground component, the image part comprising the first object, and the relative position relationship (Zhou; Fig. 8; [0131]: the foreground is separated from the background of the conference room; the corresponding output frame correctly segments the foreground).
Zhou fails to explicitly disclose:
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship.
In same field of endeavor, Goddard teaches:
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship (Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object; when the moving object 1 passes past an left end of the building 2 and further moves from the position (d) to a position (e), it is solely displayed because of the absence of the building 2 or the tree 3 there;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou to include when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship as taught by Goddard. The motivation for doing so would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65.
Regarding to claim 7 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 5, wherein
indicating a camera that captures the image to photograph the first object in a close-up manner (Zhou; [0054]: indicating whether the pixel is a foreground pixel or a background pixel ; participate in a video call are close to a camera that captures the video, e.g., in a conference room, at a desk, etc.), and
synthesizing the image of the virtual scene based on the image part comprising the first object photographed in the close-up manner (Zhou; Fig. 2; [0091]: render the foreground video; Fig. 8; [0131]: the foreground is separated from the background of the conference room; the corresponding output frame correctly segments the foreground).
Zhou fails to explicitly disclose:
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship,
In same field of endeavor, Goddard teaches:
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship (Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object; when the moving object 1 passes past an left end of the building 2 and further moves from the position (d) to a position (e), it is solely displayed because of the absence of the building 2 or the tree 3 there;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou to include when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship as taught by Goddard. The motivation for doing so would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65.
Regarding to claim 8 (Original), Zhou in view of Goddard, Creusot, and Okada discloses the method according to claim 5, wherein
adjusting the position of the foreground component, to enable an adjusted foreground component and the first object (Zhou; Fig. 8; [0131]: raise a hand with fingers separated in frame 822; and turns the hand in frame 832; the foreground is separated from the background of the conference room; the corresponding output frame correctly segments the foreground).
Zhou fails to explicitly disclose:
to satisfy the occlusion relationship;
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship,
In same field of endeavor, Goddard teaches:
to satisfy the occlusion relationship (col. 16, lines 25-37: the depth position of the moving object 1 is farther than that of the background scene 5F and accordingly invisibly hidden by any background object in the background scene; col. 17, lines 36-45: when the moving object 1 comes to a position (c) beside the tree 3, it passes on a deeper side of the tree 3 so that its portion hidden by leaves of the tree 3; col. 17, lined 50-60: the lower half portion of the moving object 1, upon being displayed, is invisibly hidden by the trunk);
when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship (Fig. 2; col. 1, lines 55-67; col. 2, lines 1-10: display background objects and a moving object; when the moving object 1 passes past an left end of the building 2 and further moves from the position (d) to a position (e), it is solely displayed because of the absence of the building 2 or the tree 3 there;
PNG
media_image7.png
300
306
media_image7.png
Greyscale
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou to include to satisfy the occlusion relationship; when a position of the first object in a background is updated, and an updated position of the first object and a position of the foreground component enable the first object and the foreground component not to satisfy the occlusion relationship as taught by Goddard. The motivation for doing so would have been to display background objects and a moving object; to display an image to pass the moving object 1 through the front of the building 2 and the tree 3 as taught by Goddard in Fig. 2 and col. 1, lines 55-67, col. 2, lines 1-10, and col. 17, lines 55-65.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
