DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
Figures 2 and 3 should be designated by a legend such as --Prior Art-- because only that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claims 3, 5, and 7 are objected to because of the following informalities: abbreviation “spp” is used in these claims. It is recommended to define the abbreviation the first time it appears in the claims. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “similar to” in claim 10 is a relative term which renders the claim indefinite. The term “similar to” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. There is no objective measure for that result.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8-10 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brownlee et al. (US 20200211265 A1), and further in view of ().
Regarding Claim 1, Brownlee discloses An information processing device comprising (¶1 reciting “an apparatus and method for performing more efficient ray tracing operations.”): a learning data acquisition processing unit that sequentially acquires ray sample data generated by ray simulation by a ray tracer from the ray tracer (¶173-184 teaching Ray Tracing with machine learning. ¶185 reciting “In one embodiment, the re-training operations performed by the machine learning engine 1600 are executed concurrently in a background process on the graphics processor unit (GPU) or host processor. ” ¶173 reciting “ray tracing is a graphics processing technique in which a light transport is simulated through physically-based rendering. ”. Ray tracing technique is obvious to a POSITA in which a light transport is simulated through a ray tracer ), and reconstructs the ray sample data and generates learning data including a student image and a teacher image for learning of an inference model. (Fig. 19 showing reconstructing the ray sample data and generating low sample count image data (corresponding to a student image) and high sample count image data (corresponding to a teacher image). ¶193 reciting “At 1903, at runtime, low sample count image frames are generated along with at least one reference region having a high sample count. At 1904, the high sample count reference region is used by the machine-learning engine and/or separate training logic (e.g., background training module 1700) to continually refine the training of the machine learning engine. ” in addition, ¶181 disclosing training an inference model, and reciting “ A machine learning engine 1500 (e.g., a CNN) receives a region of N×N pixels as high sample count image data 1702 with various per-pixel data channels such as pixel color, depth, normal, normal deviation, primitive IDs, and albedo and generates final pixel colors. Representative training data is generated using one frame's worth of low-sample count inputs 1501. The network is trained towards these inputs, generating a set of “ideal” weights 1505 which the machine learning engine 1500 subsequently uses to denoise low sample count images at runtime.”).
Claim 13 has similar limitations as of Claim(s) 1, therefore it is rejected under the same rationale as Claim(s) 1.
Claim 14 has similar limitations as of Claim(s) 1 except it is a CRM claim (Brownlee, ¶152), therefore it is rejected under the same rationale as Claim(s) 1.
Regarding Claim 2. Brownlee discloses The information processing device according to claim 1, wherein
the learning data acquisition processing unit makes resolution of the student image and that of the teacher image different by making a size of a pixel grid applied to the ray sample data vary between the student image and the teacher image. (¶182 reciting “To improve the above techniques, one embodiment of the invention augments the denoising phase to generate new training data every frame or a subset of frames (e.g., every N frames where N=2, 3, 4, 10, 25, etc). In particular, as illustrated in FIG. 16, this embodiment chooses one or more regions in each frame, referred to here as “new reference regions” 1602 which are rendered with a high sample count into a separate high sample count buffer 1604. A low sample count buffer 1603 stores the low sample count input frame 1601 (including the low sample region 1604 corresponding to the new reference region 1602).”)
Regarding Claim 8. Brownlee discloses The information processing device according to claim 1, further comprising
an online learning processing unit that uses a part of the learning data as evaluation data and evaluates appropriateness of learning on a basis of a result of comparison between an inference image acquired by an input of the student image included in the evaluation data to the inference model and the teacher image corresponding to the inference image. (¶219 reciting “ during a supervised learning training process for a neural network, the output produced by the network in response to the input representing an instance in a training data set is compared to the “correct” labeled output for that instance, an error signal representing the difference between the output and the labeled output is calculated, and the weights associated with the connections are adjusted to minimize that error as the error signal is backward propagated through the layers of the network. The network is considered “trained” when the errors for each of the outputs generated from the instances of the training data set are minimized.”)
Regarding Claim 9. Brownlee discloses The information processing device according to claim 8, wherein
the online learning processing unit performs fine tuning of a general-purpose inference model learned with general-purpose data on a basis of the learning data acquired by reconstruction of the ray sample data.
(¶184 reciting “ Regardless of how the new reference region is selected, it is used by the machine learning engine 1600 to continually refine and update the trained weights 1605 used for denoising. In particular, reference pixel colors from each new reference region 1602 and noisy reference pixel inputs from a corresponding low sample count region 1607 are rendered. Supplemental training is then performed on the machine learning engine 1600 using the high-sample-count reference region 1602 and the corresponding low sample count region 1607. In contrast to the initial training, this training is performed continuously during runtime for each new reference region 1602—thereby ensuring that the machine learning engine 1600 is precisely trained. ”)
Regarding Claim 10. Brownlee discloses The information processing device according to claim 9, wherein
the online learning processing unit extracts, from the learning data, a plurality of the student images and a plurality of the teacher images having viewpoint information similar to viewpoint information used for generation of an external output video, and performs fine tuning of the inference model by preferentially using the extracted plurality of student images and plurality of teacher images. (¶184 reciting “As in the training case (FIG. 15), the machine learning engine 1600 is trained towards a set of ideal weights 1605 for removing noise from the low sample count input frame 1601 to generate the denoised frame 1620. However, in this embodiment, the trained weights 1605 are continually updated, based on new image characteristics of new types of low sample count input frames 1601.”)
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brownlee et al. (US 20200211265 A1), and further in view of Moon et al. (US 20130077893 A1).
Regarding Claim 3. Brownlee discloses The information processing device according to claim 1.
However, Brownlee does not explicitly disclose wherein the learning data acquisition processing unit makes spp values of the student image and the teacher image different by making a degree of accumulation of the ray sample data in a frame direction vary between the student image and the teacher image.
It is well known in the art to a POSITA to have greater number of SSP in high quality image. In addition, Moon teaches “A method, system, and computer-readable storage medium are disclosed for adaptive sampling” (ABST). More specifically, ¶35 recites “Rendering of the second image may comprise performing a ray-tracing process having a greater number of samples per pixel for the high-priority regions of the second image than for other regions of the second image. As a result of the adaptive sampling guided by multilateral filtering, the second image may be less noisy than the first image.”
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the device (taught by Brownlee) to have different SPP in student image and teacher image (taught by Moon). The suggestions/motivations would have been “to reduce the noise in rendered images” (¶9), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 4. Brownlee in view of Moon discloses The information processing device according to claim 3, wherein the learning data acquisition processing unit generates the teacher image by denoising an image acquired by accumulation of the ray sample data in a case where the degree of accumulation of the ray sample data in generation of the teacher image does not satisfy an allowable standard. (Brownlee, ¶177 reciting “the noisy inputs of a frame are generated for denoising the full frame with the current network. In addition, a small region of reference pixels are generated and used for continuous training, as described below.”)
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brownlee et al. (US 20200211265 A1), and further in view of Zhang et al. (US 20240290059 A1).
Regarding Claim 11. Brownlee discloses The information processing device according to claim 1.
However, Brownlee does not explicitly disclose wherein the ray sample data is data generated for a viewport video.
Zhang teaches “A computer-implemented method of generating editable free-viewport videos ” (ABST). More specifically, ¶45 recites “the neural editing module 110 can be configured to identify rays passing through 3D bounding-boxes in a video frame (i.e., a scene) of the free-viewport video. ”
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the device (taught by Brownlee) to obtain ray sample data for a viewport video (taught by Zhang). The suggestions/motivations would have been to “to manipulate 3D bounding-boxes enclosing objects (e.g., dynamic entities) depicted in a scene of the free-viewport video. ” (¶46), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 12. Brownlee in view of Zhang discloses The information processing device according to claim 11, wherein
resolution of the teacher image matches resolution of the viewport video.
(Brownlee, Fig. 39 and ¶312 disclosing generating two types ray tracing data: lossy/lossless ray tracing data. The lossless ray tracing data would read on a teacher image that matches resolution of the input video (i.e. the viewport vide))
Allowable Subject Matter
Claims 5-7 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding Claim 5. The closest prior art Brownlee in view of Moon teaches The information processing device according to claim 3. However, the closest art fails to teach “the learning data acquisition processing unit makes the spp value of the student image match an spp value of an input image of when an external output video is generated from the input image by utilization of the inference model.” in combination with the remaining aspects of the claim and its base claim(s).
Claims 6-7 depend from Claim 5, and therefore also contain allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YI WANG whose telephone number is (571)272-6022. The examiner can normally be reached 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YI WANG/Primary Examiner, Art Unit 2619
Prosecution Insights