DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 6, 10, 15, 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al. (IBRNet: Learning Multi-View Image-Based Rendering).
In regards to claim 1, Wang teaches a method [e.g. method, see section titled “Abstract” in page 1] comprising:
receiving a request to generate a novel view of a scene, the request including a plurality of input views and a target view [e.g. Given a set of posed images of a scene, the goal of novel view synthesis is to produce photo-realistic images of the same scene at novel viewpoints, see section titled “Introduction” in page 1];
identifying a subset of the plurality of input views based on a similarity to the target view [e.g. we identify spatially nearby candidate views, then select the subset of N views whose viewing directions are most similar to the target view, see section titled “View selection and feature extraction” in page 3];
generating the novel view using the subset of the plurality of input views [e.g. The set of color images, camera projection matrices, and features for each source view forms the input for rendering a target view, see section titled “View selection and feature extraction” in page 3]; and
rendering the novel view [e.g. rendering a novel view, see section titled “View selection and feature extraction” in page 3].
In regards to claim 6, Wang teaches the method of claim 1, wherein identifying a subset of the plurality of input views based on a similarity to the target view [see rejection of claim 1 above], further comprises:
setting an in-frame term based on whether a projection of a ray falls within an image corresponding to an input view [e.g. we sample points in space and project them in the source views to fetch the corresponding colors and image features. However, the projected pixel for a sample may be located outside of the image plane. In this case, we discount this source view for this sample, see section titled “Additional implementation details” in page 12]; and
including the input view in the subset of the plurality of input views based on the in-frame term [e.g. including the source view for a sample from a plurality of sampled source views based on the sampling and projection of the points, see section titled “Additional implementation details” in page 12].
In regards to claim 10, the claim recites similar limitations as claim 1 but in the form of a non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising the method of claim 1. Furthermore, Wang teaches a non-transitory computer-readable medium storing executable instructions [e.g. computer systems inherently comprise at least one memory storing instructions, see section titled “Introduction” in page 1], which when executed by a processing device [e.g. system, see section titled “Introduction” in page 1], cause the processing device to perform operations comprising the method of claim 1. Therefore, the same rationale as claim 1 is applied.
In regards to claim 15, the claim recites similar limitations as claim 6. Therefore, the same rationale as claim 6 is applied.
In regards to claim 19, the claim recites similar limitations as claim 1 but in the form of a system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising the method of claim 1. Furthermore, Wang teaches a system comprising: a memory component; and a processing device [e.g. computer systems inherently comprise at least one memory and at least one processor, see section titled “Introduction” in page 1] coupled to the memory component, the processing device to perform operations comprising the method of claim 1. Therefore, the same rationale as claim 1 is applied.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2, 3, 11, 12, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (IBRNet: Learning Multi-View Image-Based Rendering) as applied to claims 1, 10, 19 above, and further in view of Buehler et al. (Unstructured Lumigraph Rendering).
In regards to claim 2, Wang does not explicitly teach the method of claim 1, wherein identifying a subset of the plurality of input views based on a similarity to the target view, further comprises:
calculating a camera parameter distance between each of the plurality of input views and the target view; and
selecting the subset of the plurality of input views based on the camera parameter distances.
However, Buehler teaches the method of claim 1, wherein identifying a subset of the plurality of input views based on a similarity to the target view [see rejection of claim 1 above], further comprises:
calculating a camera parameter distance [e.g. A given desired ray rd, intersects the surface proxy at some front-most point p. We consider the rays ri from p to the centers Ci of each source camera I. For each source camera we define the angular penalty, penaltyang(i), as the angular difference between ri and rd. Using these various penalties, we define the combined penalty function as penaltycomb(i). The simplified resolution penalty function that we use is penaltyres(i) = max(0, ||p – Ci|| − ||p – D||), where D is the center of the desired camera, see section titled “Camera Blending Field” in page 428] between each of the plurality of input views [e.g. ray ri of each source camera, see section titled “Camera Blending Field” in page 428] and the target view [e.g. desired ray rd, see section titled “Camera Blending Field” in page 428]; and
selecting the subset of the plurality of input views based on the camera parameter distances [e.g. we consider only the k cameras with smallest penaltyang(·)s when reconstructing a desired ray. All other cameras are assigned a weight of zero, see section titled “Camera Blending Field” in page 428].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Wang’s method with the features of
calculating a camera parameter distance between each of the plurality of input views and the target view; and
selecting the subset of the plurality of input views based on the camera parameter distances
in the same conventional manner as taught by Buehler because Buehler provides an efficient method of structured lumigraph rendering, including speed and photorealistic quality, while allowing for the use of geometric proxies, unstructured input cameras, and variations in resolution and field-of-view [see section titled “Conclusion and Future Work” in pages 430-431].
In regards to claim 3, Wang does not explicitly teach the method of claim 2, wherein the camera parameter distance is calculated based on position, orientation, and focal length parameters associated with each input view and with the target view.
However, Buehler teaches the method of claim 2, wherein the camera parameter distance is calculated based on position [e.g. The simplified resolution penalty function that we use is penaltyres(i) = max(0, ||p – Ci|| − ||p – D||), where D is the center of the desired camera, see section titled “Camera Blending Field” in page 428], orientation [e.g. angular difference, see section titled “Camera Blending Field” in page 428], and focal length [e.g. focal length, see section titled “Camera Blending Field” in page 428] parameters associated with each input view [e.g. ray ri of each source camera, see section titled “Camera Blending Field” in page 428] and with the target view [e.g. desired ray rd, see section titled “Camera Blending Field” in page 428].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Wang’s method with the features of wherein the camera parameter distance is calculated based on position, orientation, and focal length parameters associated with each input view and with the target view in the same conventional manner as taught by Buehler because Buehler provides an efficient method of structured lumigraph rendering, including speed and photorealistic quality, while allowing for the use of geometric proxies, unstructured input cameras, and variations in resolution and field-of-view [see section titled “Conclusion and Future Work” in pages 430-431].
In regards to claim 11, the claim recites similar limitations as claim 2. Therefore, the same rationale as claim 2 is applied.
In regards to claim 12, the claim recites similar limitations as claim 3. Therefore, the same rationale as claim 3 is applied.
In regards to claim 20, the claim recites similar limitations as claim 2. Therefore, the same rationale as claim 2 is applied.
Claim(s) 7, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (IBRNet: Learning Multi-View Image-Based Rendering) as applied to claims 1, 10 above, and further in view of Rong et al. (Boosting View Synthesis with Residual Transfer).
In regards to claim 7, Wang does not explicitly teach the method of claim 1, wherein generating the novel view using the subset of the plurality of input views further comprises:
learning a neural volumetric field based on the plurality of input views; predicting the novel view using the neural volumetric field;
determining residuals associated with the subset of the plurality of input views; and
combining the residuals with the predicted novel view to generate the novel view.
However, Rong teaches the method of claim 1, wherein generating the novel view using the subset of the plurality of input views further [see rejection of claim 1 above] comprises:
learning a neural volumetric field based on the plurality of input views [e.g. more recent methods, such as NeRF (neural radiance field) or NeX, replace the surface-based models with volumetric approaches where volume density varies continuously, see section titled “Volumetric View Synthesis” in page 19731];
predicting the novel view using the neural volumetric field [e.g. the output image is obtained by volume rendering. The output color at a pixel p is obtained by integrating color c and density s along the corresponding ray, see section titled “Volumetric View Synthesis” in page 19731];
determining residuals [e.g. residual errors, see section titled “Boosting View Synthesis via Residual Transfer” in page 19732] associated with the subset of the plurality of input views; and
combining the residuals with the predicted novel view to generate the novel view [e.g. The blended residual adds in just the missing detail that the baseline view synthesis method was not able to reconstruct by itself. This results in perfect reconstruction of training views and significantly improved surrounding novel views. We call this boosting the underlying view synthesis method, see section titled “Boosting View Synthesis via Residual Transfer” in page 19732].
Therefore, it would have been obvious to one of ordinary skill in the art to have modified Wang’s method with the features of
learning a neural volumetric field based on the plurality of input views; predicting the novel view using the neural volumetric field;
determining residuals associated with the subset of the plurality of input views; and
combining the residuals with the predicted novel view to generate the novel view
in the same conventional manner as taught by Rong because Rong provides a simple but effective technique to boost the rendering quality, which can be easily integrated with most view synthesis methods [see section titled “Abstract” in page 19728].
In regards to claim 16, the claim recites similar limitations as claim 7. Therefore, the same rationale as claim 7 is applied.
Allowable Subject Matter
Claims 4, 5, 8, 9, 13, 14, 17, 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 4, the prior art of record fails to teach or suggest the method of claim 1, wherein identifying a subset of the plurality of input views based on a similarity to the target view, further comprises: measuring sharpness of each input view of the plurality of input views; and down-weighting each input view of the plurality of input views based on their sharpness.
In regards to claim 5, the claim depends on at least claim 4. Therefore, claim 5 is allowed for at least the same reason as claim 4 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 8, the prior art of record fails to teach or suggest the method of claim 7, further comprising: normalizing the residuals based on an illumination channel using histogram matching.
In regards to claim 9, the claim depends on at least claim 8. Therefore, claim 9 is allowed for at least the same reason as claim 8 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 13, the claim recites similar limitations as claim 4. Therefore, claim 13 is allowed for at least the same reason as claim 4 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 14, the claim depends on at least claim 13. Therefore, claim 14 is allowed for at least the same reason as claim 13 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 17, the claim recites similar limitations as claim 8. Therefore, claim 17 is allowed for at least the same reason as claim 8 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
In regards to claim 18 the claim depends on at least claim 17. Therefore, claim 18 is allowed for at least the same reason as claim 17 if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW SHIN whose telephone number is (571)270-5764. The examiner can normally be reached Monday - Friday from 11:00AM to 7:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW SHIN/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights