DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/15/2025 was filed after the mailing date of the application. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: [ in claims 1-13: a virtual viewpoint image generation unit, a failure region acquisition unit, a re-synthesis unit, a correction information determination unit, a color synthesis weight visualization unit, a correction information propagation unit, a correction case learning unit, a learning correction unit, a quality improvement unit.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to a program per se.
Thus, in giving the term its plain meaning (see MPEP 2111.03), the claimed program is considered to include data signals per se. Data signals per se are not statutory as they fail to fall into one of the four statutory categories of invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8, 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US. Patent App. Pub. No. 2021/0368153, “Wang”, hereinafter) in view of Martin Brualla et al. (US. Patent App. Pub. No. 2022/0398705, “Brualla”, hereinafter).
As per claim 1, Wang teaches an information processing system comprising:
a virtual viewpoint image generation unit that generates a virtual viewpoint image by synthesizing multi-viewpoint images on a basis of a color synthesis weight set for each of the multi-viewpoint images according to a virtual viewpoint position (Fig. 3B, ¶ [9], synthesizing by weighting calculation on the color map of each viewpoint);
a failure region acquisition unit that receives an input of a failure region designated by a user as a region in which a failure occurs in the virtual viewpoint image (¶ [27], receiving an erroneous edge region);
a re-synthesis unit that receives a correction input of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image, corrects the color synthesis weight to a plurality of the color synthesis weights based on the correction input, and re-synthesizes a plurality of the virtual viewpoint images as a correction candidate image on a basis of the plurality of color synthesis weights (Fig. 1, step 109, ¶ [41-42], performing weighting calculation and blendering on the third color map to obtain a composite image based on the error input in step 107, ¶ [38]).
Wang does not expressly teach a correction information determination unit that receives selection information of the correction candidate image selected from a plurality of the correction candidate images as the failure is regarded as being corrected, and on a basis of the selection information, determines the color synthesis weights applied to the selected correction candidate image as correction information for correcting the failure.
Brualla teaches a similar method of synthesizing images (see Abstract), in which the method further includes the above features, i.e., a correction information determination unit that receives selection information of the correction candidate image selected from a plurality of the correction candidate images as the failure is regarded as being corrected, and on a basis of the selection information, determines the color synthesis weights applied to the selected correction candidate image as correction information for correcting the failure (¶ [93-94], and ¶ [125-126] referring to Fig. 4 and 6).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the method as taught by Brualla to the method as taught by Wang as addressed above, the advantage of which is to provide the advantage of generating synthesized images that exhibit reduced view and/or image-dependent effects and a reduced number of image artifacts (¶ [3]).
As per claim 2, the combined teachings of Wang and Brualla substantially teaches a color synthesis weight visualization unit that generates weight visualization information visualizing a magnitude of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image, wherein the re-synthesis unit receives a correction input, using the weight visualization information, of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image (see Brualla, ¶ [125], and also ¶ [30], referring to Fig. 1, virtual view 104’ based on the correction input addressed above or such as one described at ¶ [72]). Thus, claim 2 would have been obvious over the combined references for the reason above.
As per claim 3, as addressed in claim 2, the combined Wang-Brualla does also impliedly teach wherein the color synthesis weight visualization unit generates a weight map visualizing a magnitude of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image, and the re-synthesis unit receives a correction input, using the weight map, of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image (Wang, ¶ [51], Brualla, ¶ [30], ¶ [72], ¶ [125]). Thus, claim 3 would have been obvious over the combined references for the reason above.
As per claim 4, as also addressed above, the combined Wang-Brualla does also impliedly teach wherein the color synthesis weight visualization unit generates the weight map visualized with at least one of a color and a pattern according to a magnitude of the color synthesis weight of the failure region, and the re-synthesis unit receives a correction input associated with correction of at least one of the color and the pattern of the weight map (Wang, ¶ [51], Brualla, ¶ [30], ¶ [72], ¶ [125]). Thus, claim 4 would have been obvious over the combined references for the reason above.
As per claim 5, as addressed, the combined Wang-Brualla does impliedly teach wherein the color synthesis weight visualization unit visualizes, with a slidack, a magnitude of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image, and the re-synthesis unit receives a correction input, using the slidack, of the color synthesis weight of the failure region in each of the multi-viewpoint images used for synthesis of the virtual viewpoint image (Noted that the term “slidack” is not properly defined, and also not a well-known term in the art to refer to. Therefore, it is given the broadest reasonable interpretation as the correction values addressed in claims 2-4 above). Thus, claim 5 would have been obvious over the combined references for the reason above.
As per claim 6, the combined Wang-Brualla also impliedly teach a correction information propagation unit that stores the correction information determined by the correction information determination unit, wherein when the correction information is stored in the correction information propagation unit (at least temporarily for the correction process), the virtual viewpoint image generation unit generates a virtual viewpoint image by synthesizing the multi-viewpoint images using the color synthesis weights as the correction information (as addressed above). Thus, claim 6 would have been obvious over the combined references for the reason above.
As per claim 7, the combined Wang-Brualla also teaches wherein the multi-viewpoint images are generated on a basis of a shape (Brualla, ¶ [74]) and texture data of a three-dimensional model (Brualla, ¶ [77]), and the virtual viewpoint image generation unit performs, to generate a virtual viewpoint image in time series, tracking of a position of a subject in the failure region in which the correction input is made (Brualla, ¶ [85], “At least a portion of the synthesized view 250 may be determined based on output from a neural network (e.g., NN 224) using system 214 each time the user moves a head position while viewing the display and/or each time a particular image changes on the display”) on a basis of the shape and the texture data of the three- dimensional model supplied in time series to generate the multi-viewpoint images (Brualla, ¶ [110]), and generates a virtual viewpoint image by synthesizing the multi-viewpoint images using the color synthesis weights as the correction information according to a tracked position of the subject (Brualla, ¶ [85] addressed above). Thus, claim 7 would have been obvious over the combined references for the reason above.
As per claim 8, the combined Wang-Brualla impliedly teaches wherein the virtual viewpoint image generation unit specifies a type of the failure according to the tracked position of the subject (Brualla, ¶ [48], such as 2D or distortion), and generates a virtual viewpoint image by synthesizing the multi-viewpoint images using the color synthesis weights as the correction information according to the type of the failure (addressed above). Thus, claim 8 would have been obvious over the combined references for the reason above.
As per claim 10, the combined Wang-Brualla also impliedly teaches wherein the tracking includes Mesh-tracking (at best understood by the examiner as the mesh proxy geometry described at ¶ [74] of Brualla). Thus, claim 10 would have been obvious over the combined references for the reason above.
As per claim 11, as addressed in claim 7, the combined Wang-Brualla does also teach a correction case learning unit that acquires the shape and the texture data of the three-dimensional model and the correction information as correction cases, and learns learning information for realizing detecting a failure region of a virtual viewpoint image generated by synthesizing multi-viewpoint images generated from the shape and texture data of the three-dimensional model and correction of the detected failure region by learning using the correction cases (as addressed in claim 7. The correction case learning unit is interpreted as the neural network described at ¶ [91], referring to Fig. 4 of Brualla. Thus, claim 11 would have been obvious over the combined references for the reason above.
As per claim 12, as addressed in claim 11, the combined Wang-Brualla does also impliedly teach a learning correction unit that detects the failure region on a basis of the learning information for a virtual viewpoint image generated by synthesizing the multi-viewpoint images and generated by the virtual viewpoint image generation unit, and corrects the detected failure region. Thus, claim 12 would have been obvious over the combined references for the reason above.
As per claim 13, the combined Wang-Brualla does also teach a quality improvement unit that improves quality of the multi-viewpoint images when the multi-viewpoint images used to generate the virtual viewpoint image have quality lower than a predetermined level of quality (Brualla, ¶ [31]). Thus, claim 13 would have been obvious over the combined references for the reason above.
Claim 14, which is similar in scope to claim 1 as addressed above, is thus rejected under the same rationale.
Claim 15, which is similar in scope to claim 1 as addressed above, is thus rejected under the same rationale.
Allowable Subject Matter
Claim 9 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art taken singly or in combination does not teach or suggest, an information processing system, among other things, comprising:
…wherein the type of the failure includes a failure caused by low quality of the multi-viewpoint images having the large color synthesis weights, a failure in which a background is reflected in a portion of a foreground, and a failure in which a foreground is reflected in a portion of a background.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hau H. Nguyen whose telephone number is: 571-272-7787. The examiner can normally be reached on MON-FRI from 8:30-5:30.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard, can be reached on (571) 272-7773.
The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/HAU H NGUYEN/Primary Examiner, Art Unit 2611
Prosecution Insights