Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed on 7 August 2026 have been fully considered but they are not persuasive.
Claims 1-20 are pending in this application and have been considered below.
Argument:
The applicant argues that Yu is directed to generating a virtual-viewpoint image from original-viewpoint image information and depth information. Applicant argues that Yu determines a virtual viewpoint and an imaging plane based on pose information of an original viewpoint, then determines a search-step range of rays from the virtual viewpoint to pixels of the imaging plane based on depth information, then searches along those rays to determine reconstruction points, and then colors the reconstruction points based on color information of the original- viewpoint image to generate the virtual-viewpoint image. Applicant then admits that Yu also discusses depth-fusion and visibility/weighting operations used in this virtual-viewpoint synthesis process.
Thus, Applicant argues that Yu's data flow is centered on real-scene or multi-view image/depth data and a synthetic camera viewpoint. Applicant argues that Yu's virtual-viewpoint image is an image of a scene from a virtual viewpoint generated by reconstruction and coloring of scene points. Applicant concludes that Yu does not use a target object in a real scene to determine a pose of an associated virtual object, does not adjust a virtual model corresponding to that virtual object based on the virtual-object pose, and does not rasterize the adjusted virtual model to obtain a depth map corresponding to the virtual object.
Applicant then argues that Yu's "virtual viewpoint image" is generated by determining a virtual viewpoint and imaging plane, searching rays based on depth information, determining reconstruction points, and coloring those reconstruction points using original-viewpoint color information. Applicant concludes that Yu’s disclosure is not the same as generating a virtual image corresponding to a virtual object by adjusting a virtual model corresponding to the virtual object based on the virtual-object pose and rasterizing the adjusted virtual model to obtain a virtual-object depth map. Applicants argue that in Yu, the depth information and ray-search operations relate to scene reconstruction from viewpoint data; they are not a rasterized depth map of an adjusted virtual model corresponding to an associated virtual object.
Applicants thus conclude that the distinction is not merely one of terminology. The applicants argue that amended claim requires a model-to-depth-map data path for a virtual object associated with a target object. Applicants contend that Yu's cited process is a multi-view or virtual-viewpoint synthesis process for scene data.
Response:
Our reviewing Court has made clear that examined claims are interpreted as broadly as is reasonable using ordinary and accustomed term meanings so as to be consistent with the Specification. In re Thrift, 298 F.3d 1357, 1364 (Fed. Cir. 2002). The Court further has explained that the interpretations are to be made while “taking into account whatever enlightenment by way of definitions or otherwise that may be afforded by the written description contained in the applicant’s specification,” In re Morris, 127 F.3d 1048, 1054 (Fed. Cir. 1997), but without reading limitations from examples given in the Specification into the claims, In re Zletz, 893 F.2d 319, 321-22 (Fed. Cir. 1989). Moreover, there is no ipsissimis verbis test for determining whether a reference discloses a claim element, i.e., identity of terminology is not required. In re Bond, 910 F.2d 831, 832 (Fed. Cir. 1990).
In this case, the broadest reasonable interpretation of the claims includes re-arranging slightly, or using options explained and anticipated by a reference. If the reference teaches additional steps, those are accounted for as the claims use “comprising,” such that additional steps would be accounted for and considered to be infringing, and thus must be rejected over.
Thus, the examiner maintains that Chinese Patent Publication 116385577 A, (Yu et al.) shows the limitation target object ("based on the point of the target distance function value on each light generating symbol change, determining the reconstruction point on each light," page 3, paragraph 9, where the "light generating symbol change" is a target object) and wherein generating the virtual image corresponding to the virtual object comprises adjusting a virtual model corresponding to the virtual object based on the pose of the virtual object ("through the pose information of the original viewpoint image, depth information and color information for multi-directional comprehensive analysis, constructing the light between the virtual viewpoint and the imaging plane and solving the coordinate and color of the reconstruction point on each light, finally generating the virtual viewpoint image," page 5, paragraph 4) and rasterizing the adjusted virtual model to obtain a depth map corresponding to the virtual object ("based on the pixel coordinate system corresponding to the depth map of the original viewpoint image, performing coordinate transformation on each pixel point on the imaging plane," page 3, paragraph 5, where rasterizing is coordinate transformation on each pixel point).
Argument:
The applicant argues that Gao is directed to image encoding/decoding and improving bit efficiency by applying filter processing to an input image and encoding the resulting image. Applicants note that the Office action relies on Gao, including paragraph [0083], for a general teaching that filters such as low-pass filters, Gaussian filters, smoothing filters, averaging filters, bilateral filters, median filters, and similar filters may be used. Applicants conclude that Gao therefore provides, at most, a general image-filtering or smoothing teaching in an encoding/decoding context.
Applicants thus conclude that Gao does not remedy the deficiencies of Yu. Applicants state that Gao does not disclose determining a pose of a virtual object associated with a target object, adjusting a virtual model corresponding to that virtual object based on the virtual-object pose, or rasterizing the adjusted virtual model to obtain a depth map corresponding to the virtual object. Thus, applicants conclude that since Gao is cited only for generic smoothing or filtering in an encoding/decoding context, that it does not supply the missing virtual-model adjustment and rasterization limitation.
Response:
US Patent Publication 2025 0024033 A1, (Gao et al.) is relied upon to show the limitation “performing smoothing.” Since Applicants are not arguing the patentability of the feature “smoothing filter,” this argument does not currently impact patentability.
Priority
Receipt is acknowledged that application claims priority to foreign application with application number CN 2023 10889513.1 dated 19 July 2026. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Specification - Title
The title of the invention has been amended. The objection to the title is withdrawn.
Claim Rejections - 35 USC § 101
Claims 1-20 were rejected under 35 U.S.C. 101 because the claimed invention was directed to an abstract idea without significantly more. Claims 1, 11 and 20 have been amended. The rejection of claims 1-20 under 35 U.S.C. 101 is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 103 as obvious over Chinese Patent Publication 116385577 A, (Yu et al.) in view of US Patent Publication 2025 0024033 A1, (Gao et al.). The references are listed in a PTO-892 from the Office Action in which they are first used.
Claim 1
[AltContent: textbox (Yu et al. Fig. 3, showing determining distances in an image.)]
PNG
media_image1.png
357
397
media_image1.png
Greyscale
Regarding Claim 1, Yu et al. teach a method of image processing ("image processing technology field, especially relates to a virtual viewpoint image generating method and device," page 2, 1st paragraph), comprising:
determining a pose of a virtual object associated with a target object ("based on the point of the target distance function value on each light generating symbol change, determining the reconstruction point on each light," page 3, paragraph 9, where the "light generating symbol change" is a target object) based on a pose of the target object in a real scene ("based on pose information of the original viewpoint image, determining the imaging plane of the virtual viewpoint and virtual viewpoint," page 17, 5th paragraph);
generating a virtual image corresponding to the virtual object based on the pose of the virtual object ("A virtual viewpoint image generating method according to the present invention," page 3, paragraph 6, and "obtaining the depth value of each pixel point after coordinate transformation in the original viewpoint image, determining the range of the depth value as the search step range of the light ray between each pixel point on the virtual viewpoint to the imaging plane'" page 3, paragraph 5, where each pixel is an object), wherein generating the virtual image corresponding to the virtual object comprises adjusting a virtual model corresponding to the virtual object based on the pose of the virtual object ("through the pose information of the original viewpoint image, depth information and color information for multi-directional comprehensive analysis, constructing the light between the virtual viewpoint and the imaging plane and solving the coordinate and color of the reconstruction point on each light, finally generating the virtual viewpoint image," page 5, paragraph 4) and rasterizing the adjusted virtual model to obtain a depth map corresponding to the virtual object ("based on the pixel coordinate system corresponding to the depth map of the original viewpoint image, performing coordinate transformation on each pixel point on the imaging plane," page 3, paragraph 5, where rasterizing is coordinate transformation on each pixel point); and
obtaining a first target image based on the virtual image and a real scene image corresponding to the real scene ("each pixel on the virtual viewpoint image on the target three-dimensional fusion surface corresponding to the position of the point has been determined, then according to the colour information of the original viewpoint image to calculate the colour value of each reconstruction point pixel to obtain the final virtual view," page 7, last paragraph).
[AltContent: textbox (Gao et al. Fig. 3, showing breaking an image into targets.)]
PNG
media_image2.png
300
549
media_image2.png
Greyscale
Yu et al. is not relied upon to explicitly teach all of obtain a second target image.
However, Gao et al. teach performing smoothing on the first target image to obtain a second target image ("The filters 40A to 40C corresponding to the machine tasks include at least one of a noise removal filter, a sharpening filter, a bit depth conversion filter, a color space conversion filter, a resolution conversion filter, and a filter using a neural network. The noise removal filter includes at least one of a low-pass filter, a Gaussian filter, a smoothing filter, an averaging filter, a bilateral filter, and a median filter to remove noise by reducing information on details of the input image," paragraph [0083] where the second image is created by removing noise).
Therefore, taking the teachings of Yu et al. and Gao et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “Virtual Viewpoint Image Generating Method and Device” as taught by Yu et al. to use Image encoding method as taught by Gao et al. The suggestion/motivation for doing so would have been that, “An object of the present disclosure is to improve bit efficiency of a bit stream transmitted from an image encoding device to an image decoding device.” as noted by the Gao et al. disclosure in paragraph [0006], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that various techniques will work better with various images and situations; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of apparatus claim 11 and apparatus claim 20 while noting that the rejection above cites to both device and method disclosures. Claims 11 and 20 are mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Yu et al. teach the method of claim 1, wherein the performing smoothing on the first target image comprises: performing local smoothing on a boundary of the virtual object in the first target image ("the solution of the present embodiment is to use the PCF algorithm to smooth the sawtooth by sampling. The basic idea of the PCF algorithm in this embodiment is: using the sampling method, sampling the pixel near the specific pixel of Shadow Map, and taking the average value, so as to obtain a number of O to 1, taking it as the weight to calculate so that the sawtooth at the edge is smoother," page 12, last paragraph).
Claim 3
Regarding claim 3, Yu et al. teach the method of claim 2, wherein the performing smoothing on the first target image further comprises: performing overall smoothing on the first target image after the local smoothing ("performing the boundary expansion, then combining the expansion and calculating wj according to the idea, the operation can make the edge part of the final view image more smooth," page 23, 2nd paragraph).
Claim 4
Regarding claim 4, Yu et al. teach the method of claim 2, wherein the performing local smoothing on a boundary of the virtual object in the first target image comprises: determining a region to be smoothed containing a boundary of the virtual image in the first target image based on the boundary ("the solution of the present embodiment is to use the PCF algorithm to smooth the sawtooth by sampling. The basic idea of the PCF algorithm in this embodiment is: using the sampling method, sampling the pixel near the specific pixel of Shadow Map, and taking the average value, so as to obtain a number of O to 1, taking it as the weight to calculate so that the sawtooth at the edge is smoother," page 12, last paragraph); and
determining a target depth value of each pixel in the region to be smoothed, wherein the determined target depth value of each pixel is as close as possible to an original depth value of the pixel and target depth values of adjacent pixels of the pixel as a whole ("The depth information of the original viewpoint image in this embodiment comprises a plurality of depth maps, for each pixel on the imaging plane, by coordinate conversion, finding each depth map corresponding to the depth value of the pixel coordinate system, calculating the depth value and calculating the result, and expanding the result to a certain degree, so as to obtain the effective search step range of the light corresponding to the pixel," page 7, 3rd paragraph).
Claim 5
Regarding claim 5, Yu et al. teach the method of claim 4, wherein the determining a target depth value of each pixel in the region to be smoothed comprises: constructing an energy function based on an accumulation of a square of a difference between the target depth value and the original depth value of each pixel, and
an accumulation of squares of differences between the target depth value of each pixel and the target depth values of the adjacent pixels ("calculating the cloud normal algorithm is directly calculating the normal direction the point cloud, the method specifically comprises: polynomial curved surface local fitting algorithm, least squares plane fitting estimation algorithm. The embodiment uses the least squares plane fitting estimation algorithm, the method calculating time is short, it can satisfy the requirement of the method in real time," page 13, 3rd paragraph); and
obtaining the target depth value for each pixel by minimizing the energy function ("the pre-processing module captures five paths of color video stream captured by the color camera, finishing the extraction of the portrait part by the real-time matting algorithm based on neural network; the algorithm uses two neural network, the basic network calculates the low resolution result, the result is run on the selective patch with high resolution by the second network," page 22, 2nd paragraph, where a neural network works by minimizing an energy function).
Claim 6
Regarding claim 6, Yu et al. teach the method of claim 4, wherein the adjacent pixels comprise a pixel adjacent in a first direction and a pixel adjacent in a second direction, and the first direction and the second direction are perpendicular to each other ("the analysis module starts from the starting point of the effective search stepping range of each light, for a point p on each light, calculating the fusion weight wj, j belongs to (1, 2, 3, 4) around the pixel around the pixel, so as to further reduce the influence of high variance noise on the algorithm," page 23, 2nd paragraph, where 1,3 is a first and 2, 4 is a second direction, perpendicular to each other).
Claim 7
Regarding claim 7, Yu et al. teach the method of claim 5, wherein in the energy function, the squares of the differences between the target depth values of the adjacent pixels correspond to respective weight values, which are determined based on color deviations among the adjacent pixels ("calculating the cloud normal algorithm is directly calculating the normal direction the point cloud, the method specifically comprises: polynomial curved surface local fitting algorithm, least squares plane fitting estimation algorithm. The embodiment uses the least squares plane fitting estimation algorithm, the method calculating time is short, it can satisfy the requirement of the method in real time," page 13, 3rd paragraph, and "projecting to the color image to realize the alignment of the depth map and color image, reducing the influence caused by the depth camera and color camera space position deviation" page 15, 4th paragraph).
Claim 8
Regarding claim 8, Yu et al. teach the method of claim 3, as noted above.
Yu et al. is not relied upon to explicitly teach all of gaussian filtering.
However, Gao et al. teach wherein the overall smoothing comprises Gaussian filtering processing ("The noise removal filter includes at least one of a low-pass filter, a Gaussian filter, a smoothing filter, an averaging filter, a bilateral filter, and a median filter to remove noise by reducing information on details of the input image," paragraph [0083]).
Yu et al. and Gao et al. are combined as per claim 1.
Claim 9
Regarding claim 9, Yu et al. teach the method of claim 1, wherein the determining a pose of a virtual object associated with a target object based on a pose of the target object in a real scene comprises:
determining a pose and a state of a virtual object based on a pose and a state of the target object in a real scene ("based on the pose information of the original viewpoint image, determining the imaging plane of the virtual viewpoint and the virtual viewpoint," page 2, last paragraph); and
the generating a virtual image corresponding to the virtual object based on the pose of the virtual object comprises:
generating the virtual image corresponding to the virtual object based on the pose and the state of the virtual object ("based on the depth information of the original viewpoint image, determining the search step range of the light between the virtual viewpoint to each pixel point on the imaging plane; searching each light based on the search step range, determining the reconstruction point on each light; coloring the reconstruction point on each light based on the color information of the original viewpoint image, generating a virtual viewpoint image," page 3, 1st paragraph).
Claim 10
Regarding claim 10, Yu et al. teach the method of claim 1, further comprising: performing, based on the second target image, 3-Dimentional reconstruction and texture mapping and rendering ("at the same time, by converting the 2 D point on the depth map to the 3 D point in the world coordinate system, and then projecting to the color image to realize the alignment of the depth map and color image, reducing the influence caused by the depth camera and color camera space position deviation," page 15, 4th paragraph).
Claim 11
Regarding claim 11, Yu et al. teach an electronic device, comprising: at least one memory and at least one processor ("an electronic device, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, when the processor executes the program," page 5, 1st paragraph);
wherein the memory is configured to store program codes, and the processor is configured to call the program codes stored in the memory to cause the electronic device to execute acts ("image processing technology field, especially relates to a virtual viewpoint image generating method and device," page 2, 1st paragraph) comprising:
determining a pose of a virtual object associated with a target object ("based on the point of the target distance function value on each light generating symbol change, determining the reconstruction point on each light," page 3, paragraph 9, where the "light generating symbol change" is a target object) based on a pose of the target object in a real scene("based on pose information of the original viewpoint image, determining the imaging plane of the virtual viewpoint and virtual viewpoint," page 17, 5th paragraph);
generating a virtual image corresponding to the virtual object based on the pose of the virtual object ("A virtual viewpoint image generating method according to the present invention," page 3, paragraph 6, and "obtaining the depth value of each pixel point after coordinate transformation in the original viewpoint image, determining the range of the depth value as the search step range of the light ray between each pixel point on the virtual viewpoint to the imaging plane'" page 3, paragraph 5, where each pixel is an object), wherein generating the virtual image corresponding to the virtual object comprises adjusting a virtual model corresponding to the virtual object based on the pose of the virtual object ("through the pose information of the original viewpoint image, depth information and color information for multi-directional comprehensive analysis, constructing the light between the virtual viewpoint and the imaging plane and solving the coordinate and color of the reconstruction point on each light, finally generating the virtual viewpoint image," page 5, paragraph 4) and rasterizing the adjusted virtual model to obtain a depth map corresponding to the virtual object ("based on the pixel coordinate system corresponding to the depth map of the original viewpoint image, performing coordinate transformation on each pixel point on the imaging plane," page 3, paragraph 5, where rasterizing is coordinate transformation on each pixel point); and
obtaining a first target image based on the virtual image and a real scene image corresponding to the real scene ("each pixel on the virtual viewpoint image on the target three-dimensional fusion surface corresponding to the position of the point has been determined, then according to the colour information of the original viewpoint image to calculate the colour value of each reconstruction point pixel to obtain the final virtual view," page 7, last paragraph).
Yu et al. is not relied upon to explicitly teach all of obtain a second target image.
However, Gao et al. teach performing smoothing on the first target image to obtain a second target image ("The filters 40A to 40C corresponding to the machine tasks include at least one of a noise removal filter, a sharpening filter, a bit depth conversion filter, a color space conversion filter, a resolution conversion filter, and a filter using a neural network. The noise removal filter includes at least one of a low-pass filter, a Gaussian filter, a smoothing filter, an averaging filter, a bilateral filter, and a median filter to remove noise by reducing information on details of the input image," paragraph [0083] where the second image is created by removing noise).
Yu et al. and Gao et al. are combined as per claim 1.
Claim 12
Regarding claim 12, Yu et al. teach the device of claim 11, wherein the performing smoothing on the first target image comprises: performing local smoothing on a boundary of the virtual object in the first target image ("the solution of the present embodiment is to use the PCF algorithm to smooth the sawtooth by sampling. The basic idea of the PCF algorithm in this embodiment is: using the sampling method, sampling the pixel near the specific pixel of Shadow Map, and taking the average value, so as to obtain a number of O to 1, taking it as the weight to calculate so that the sawtooth at the edge is smoother," page 12, last paragraph).
Claim 13
Regarding claim 13, Yu et al. teach the device of claim 12, wherein the performing smoothing on the first target image further comprises: performing overall smoothing on the first target image after the local smoothing ("performing the boundary expansion, then combining the expansion and calculating wj according to the idea, the operation can make the edge part of the final view image more smooth," page 23, 2nd paragraph).
Claim 14
Regarding claim 14, Yu et al. teach the device of claim 12, wherein the performing local smoothing on a boundary of the virtual object in the first target image comprises: determining a region to be smoothed containing a boundary of the virtual image in the first target image based on the boundary ("the solution of the present embodiment is to use the PCF algorithm to smooth the sawtooth by sampling. The basic idea of the PCF algorithm in this embodiment is: using the sampling method, sampling the pixel near the specific pixel of Shadow Map, and taking the average value, so as to obtain a number of O to 1, taking it as the weight to calculate so that the sawtooth at the edge is smoother," page 12, last paragraph); and
determining a target depth value of each pixel in the region to be smoothed, wherein the determined target depth value of each pixel is as close as possible to an original depth value of the pixel and target depth values of adjacent pixels of the pixel as a whole ("The depth information of the original viewpoint image in this embodiment comprises a plurality of depth maps, for each pixel on the imaging plane, by coordinate conversion, finding each depth map corresponding to the depth value of the pixel coordinate system, calculating the depth value and calculating the result, and expanding the result to a certain degree, so as to obtain the effective search step range of the light corresponding to the pixel," page 7, 3rd paragraph).
Claim 15
Regarding claim 15, Yu et al. teach the device of claim 14, wherein the determining a target depth value of each pixel in the region to be smoothed comprises: constructing an energy function based on an accumulation of a square of a difference between the target depth value and the original depth value of each pixel, and an accumulation of squares of differences between the target depth value of each pixel and the target depth values of the adjacent pixels ("calculating the cloud normal algorithm is directly calculating the normal direction the point cloud, the method specifically comprises: polynomial curved surface local fitting algorithm, least squares plane fitting estimation algorithm. The embodiment uses the least squares plane fitting estimation algorithm, the method calculating time is short, it can satisfy the requirement of the method in real time," page 13, 3rd paragraph); and
obtaining the target depth value for each pixel by minimizing the energy function ("the pre-processing module captures five paths of color video stream captured by the color camera, finishing the extraction of the portrait part by the real-time matting algorithm based on neural network; the algorithm uses two neural network, the basic network calculates the low resolution result, the result is run on the selective patch with high resolution by the second network," page 22, 2nd paragraph, where a neural network works by minimizing an energy function).
Claim 16
Regarding claim 16, Yu et al. teach the device of claim 14, wherein the adjacent pixels comprise a pixel adjacent in a first direction and a pixel adjacent in a second direction, and the first direction and the second direction are perpendicular to each other ("the analysis module starts from the starting point of the effective search stepping range of each light, for a point p on each light, calculating the fusion weight wj, j belongs to (1, 2, 3, 4) around the pixel around the pixel, so as to further reduce the influence of high variance noise on the algorithm," page 23, 2nd paragraph, where 1,3 is a first and 2, 4 is a second direction, perpendicular to each other).
Claim 17
Regarding claim 17, Yu et al. teach the device of claim 15, wherein in the energy function, the squares of the differences between the target depth values of the adjacent pixels correspond to respective weight values, which are determined based on color deviations among the adjacent pixels ("calculating the cloud normal algorithm is directly calculating the normal direction the point cloud, the method specifically comprises: polynomial curved surface local fitting algorithm, least squares plane fitting estimation algorithm. The embodiment uses the least squares plane fitting estimation algorithm, the method calculating time is short, it can satisfy the requirement of the method in real time," page 13, 3rd paragraph, and "projecting to the color image to realize the alignment of the depth map and color image, reducing the influence caused by the depth camera and color camera space position deviation" page 15, 4th paragraph).
Claim 18
Regarding claim 18, Yu et al. teach the device of claim 13, as noted above.
Yu et al. is not relied upon to explicitly teach all of gaussian filtering.
However, Gao et al. teach wherein the overall smoothing comprises Gaussian filtering processing ("The noise removal filter includes at least one of a low-pass filter, a Gaussian filter, a smoothing filter, an averaging filter, a bilateral filter, and a median filter to remove noise by reducing information on details of the input image," paragraph [0083]).
Yu et al. and Gao et al. are combined as per claim 1.
Claim 19
Regarding claim 19, Yu et al. teach the device of claim 11, wherein the determining a pose of a virtual object associated with a target object based on a pose of the target object in a real scene comprises: determining a pose and a state of a virtual object based on a pose and a state of the target object in a real scene ("based on the pose information of the original viewpoint image, determining the imaging plane of the virtual viewpoint and the virtual viewpoint," page 2, last paragraph); and
the generating a virtual image corresponding to the virtual object based on the pose of the virtual object comprises: generating the virtual image corresponding to the virtual object based on the pose and the state of the virtual object ("based on the depth information of the original viewpoint image, determining the search step range of the light between the virtual viewpoint to each pixel point on the imaging plane; searching each light based on the search step range, determining the reconstruction point on each light; coloring the reconstruction point on each light based on the color information of the original viewpoint image, generating a virtual viewpoint image," page 3, 1st paragraph).
Claim 20
Regarding claim 20, Yu et al. teach a non-transitory computer storage medium, wherein the non-transitory computer storage medium stores program codes ("an electronic device, comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, when the processor executes the program," page 5, 1st paragraph), wherein the program codes, when executed by a computer device, cause the computer device to execute acts ("image processing technology field, especially relates to a virtual viewpoint image generating method and device," page 2, 1st paragraph) comprising:
determining a pose of a virtual object associated with a target object ("based on the point of the target distance function value on each light generating symbol change, determining the reconstruction point on each light," page 3, paragraph 9, where the "light generating symbol change" is a target object) based on a pose of the target object in a real scene ("based on pose information of the original viewpoint image, determining the imaging plane of the virtual viewpoint and virtual viewpoint," page 17, 5th paragraph);
generating a virtual image corresponding to the virtual object based on the pose of the virtual object ("A virtual viewpoint image generating method according to the present invention," page 3, paragraph 6, and "obtaining the depth value of each pixel point after coordinate transformation in the original viewpoint image, determining the range of the depth value as the search step range of the light ray between each pixel point on the virtual viewpoint to the imaging plane'" page 3, paragraph 5, where each pixel is an object), wherein generating the virtual image corresponding to the virtual object comprises adjusting a virtual model corresponding to the virtual object based on the pose of the virtual object ("through the pose information of the original viewpoint image, depth information and color information for multi-directional comprehensive analysis, constructing the light between the virtual viewpoint and the imaging plane and solving the coordinate and color of the reconstruction point on each light, finally generating the virtual viewpoint image," page 5, paragraph 4) and rasterizing the adjusted virtual model to obtain a depth map corresponding to the virtual object ("based on the pixel coordinate system corresponding to the depth map of the original viewpoint image, performing coordinate transformation on each pixel point on the imaging plane," page 3, paragraph 5, where rasterizing is coordinate transformation on each pixel point);
obtaining a first target image based on the virtual image and a real scene image corresponding to the real scene ("each pixel on the virtual viewpoint image on the target three-dimensional fusion surface corresponding to the position of the point has been determined, then according to the colour information of the original viewpoint image to calculate the colour value of each reconstruction point pixel to obtain the final virtual view," page 7, last paragraph); and
Yu et al. is not relied upon to explicitly teach all of obtain a second target image.
However, Gao et al. teach performing smoothing on the first target image to obtain a second target image ("The filters 40A to 40C corresponding to the machine tasks include at least one of a noise removal filter, a sharpening filter, a bit depth conversion filter, a color space conversion filter, a resolution conversion filter, and a filter using a neural network. The noise removal filter includes at least one of a low-pass filter, a Gaussian filter, a smoothing filter, an averaging filter, a bilateral filter, and a median filter to remove noise by reducing information on details of the input image," paragraph [0083] where the second image is created by removing noise).
Yu et al. and Gao et al. are combined as per claim 1.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
International Patent Publication 2018 099557 A1 to Bergen et al. discloses producing in real-time a digital composite image from a sequence of digital images recorded by a camera device, in particular an endoscopic camera device, the image processing device comprising a selecting unit, a key point detection unit, a transforming unit and a joining unit, wherein the key point detection unit comprises a filter configured for producing a filter response for the reference image and for producing a filter response for the further image, wherein the key point detection unit comprises a maximum detection unit configured for executing following steps separately for the filter response for the reference image.
US Patent Publication 2023 0401799 A1 to Li et al. discloses an augmented reality method, to dynamically associate an orientation of a virtual object with an orientation of a real object. The method in this application includes: obtaining a target image shot by a camera and first location information of a first object in the target image; obtaining second location information of a second object in a three-dimensional coordinate system and third location information of a third object in the three-dimensional coordinate system.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATH E WELLS whose telephone number is (703)756-4696. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Heath E. Wells/Examiner, Art Unit 2664
Date: 4 October 2026
Prosecution Insights