DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3 – 5, 8, 10 – 12, 15, and 17 – 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al. (US 20240135513) in view of Nepveu et al. (KR 20230116765), and Kopp et al. (US 20240185518).
Regarding claim 1.
Singh teaches:
A method for generating a simulated scene, comprising:
generating, by a first diffusion network, a first key frame based on a text description input and a high definition (HD) map input (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).);
generating, by a second diffusion network, a second key frame based on the text description input, the HD map input, and the warped first key frame (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).); and
Singh fails to teach:
warping the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
generating, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Nepveu teaches:
warping the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
Kopp teaches:
generating, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 3.
Singh, Nepveu, and Kopp teach:
The method of claim 1, further comprising:
generating a trajectory between the first key frame and the second key frame, wherein the middle frame is generated at a point along the trajectory (Kopp [0493] In an example, processing logic may provide an option to keep one jaw/dental arch fixed on the screen, and may only apply a relative movement to the other jaw based on a selected image. This may enable a doctor or patient to focus on a specific jaw for a 3D scene fixed on a screen and observe how the other jaw moves relative to the fixed jaw. For example, processing logic may provide functionality of a virtual articulator model or jaw motion device, where a movement trajectory is dictated by the selected images.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 4.
Singh, Nepveu, and Kopp teach:
The method of claim 1, further comprising:
applying a warp frame to enforce consistency between the first key frame and the second key frame (Kopp [0324] In embodiments, the modified frame generator 336 performs a warp in the feature space based on the received optical flow (which may also be in the feature space). The modified frame generator 336 may generate modified frames with post-treatment or post-alteration teeth in a manner that reduces flow loss (e.g., perceptual correctness loss in feature space) and/or affine regularization loss for optical flow.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 5.
Singh, Nepveu, and Kopp teach:
The method of claim 1,
wherein the first diffusion network, the second diffusion network, and the third diffusion network share weights (Kopp [0335] Training of a neural network may be achieved in a supervised learning manner, which involves feeding a training dataset consisting of labeled inputs through the network, observing its outputs, defining an error (by measuring the difference between the outputs and the label values), and using techniques such as deep gradient descent and backpropagation to tune the weights of the network across all its layers and nodes such that the error is minimized. [0340] The next layer is called a hidden layer, and nodes at the hidden layer each receive one or more of the input values. Each node contains parameters (e.g., weights) to apply to the input values. Each node therefore essentially inputs the input values into a multivariate function (e.g., a non-linear mathematical transformation) to produce an output value. A next layer may be another hidden layer or an output layer. In either case, the nodes at the next layer receive the output values from the nodes at the previous layer, and each node applies weights to those values and then generates its own output value.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 8.
Singh teaches:
A system for generating a simulated scene, comprising: a memory storing instructions (Singh [0843] Each of the components 8202-8216 of the scene-based image editing system 106 optionally include software, hardware, or both. For example, the components 8202-8216 include one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices, such as a client device or server device. When executed by the one or more processors, the computer-executable instructions of the scene-based image editing system 106 cause the computing device(s) to perform the methods described herein.);
And a processor configured to execute the instructions to: generate, by a first diffusion network, a first key frame based on a text description input and a high definition (HD) map input (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).);
generate, by a second diffusion network, a second key frame based on the text description input, the HD map input, and the warped first key frame (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).); and
Singh fails to teach:
warp the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
generate, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Nepveu teaches:
warp the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
Kopp teaches:
generate, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 10.
Singh, Nepveu, and Kopp teach:
The system of claim 8, wherein the processor is further configured to:
generate a trajectory between the first key frame and the second key frame, wherein the middle frame is generated at a point along the trajectory (Kopp [0493] In an example, processing logic may provide an option to keep one jaw/dental arch fixed on the screen, and may only apply a relative movement to the other jaw based on a selected image. This may enable a doctor or patient to focus on a specific jaw for a 3D scene fixed on a screen and observe how the other jaw moves relative to the fixed jaw. For example, processing logic may provide functionality of a virtual articulator model or jaw motion device, where a movement trajectory is dictated by the selected images.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 11.
Singh, Nepveu, and Kopp teach:
The system of claim 8, wherein the processor is further configured to:
apply a warp frame to enforce consistency between the first key frame and the second key frame (Kopp [0324] In embodiments, the modified frame generator 336 performs a warp in the feature space based on the received optical flow (which may also be in the feature space). The modified frame generator 336 may generate modified frames with post-treatment or post-alteration teeth in a manner that reduces flow loss (e.g., perceptual correctness loss in feature space) and/or affine regularization loss for optical flow.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 12.
Singh, Nepveu, and Kopp teach:
The system of claim 8,
wherein the first diffusion network, the second diffusion network, and the third diffusion network share weights (Kopp [0335] Training of a neural network may be achieved in a supervised learning manner, which involves feeding a training dataset consisting of labeled inputs through the network, observing its outputs, defining an error (by measuring the difference between the outputs and the label values), and using techniques such as deep gradient descent and backpropagation to tune the weights of the network across all its layers and nodes such that the error is minimized. [0340] The next layer is called a hidden layer, and nodes at the hidden layer each receive one or more of the input values. Each node contains parameters (e.g., weights) to apply to the input values. Each node therefore essentially inputs the input values into a multivariate function (e.g., a non-linear mathematical transformation) to produce an output value. A next layer may be another hidden layer or an output layer. In either case, the nodes at the next layer receive the output values from the nodes at the previous layer, and each node applies weights to those values and then generates its own output value.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 15.
Singh teaches:
A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for generating a simulated scene, the method comprising (Singh [0845] FIGS. 1-82, the corresponding text, and the examples provide a number of different methods, systems, devices, and non-transitory computer-readable media of the scene-based image editing system 106.):
generating, by a first diffusion network, a first key frame based on a text description input and a high definition (HD) map input (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).);
generating, by a second diffusion network, a second key frame based on the text description input, the HD map input, and the warped first key frame (Singh [0382] The following provides a more detailed description of the text-guided visual feature manipulation. As previously mentioned, in one or more embodiments, the scene-based image editing system 106 utilizes the joint embedding space 1816 to manipulate the visual feature maps 1810 with the text instructions of the modification input 1804a-1804b via vector arithmetic operations. When manipulating certain objects or object attributes, the object modification neural network 1806 aims to modify only specific regions while keeping other regions unchanged. Accordingly, the object modification neural network 1806 conducts vector arithmetic operations between the visual feature maps 1810 represented as V∈custom-character.sup.1024×7×7 and the textual features 1814a-1814b (e.g., represented as textual feature vectors).); and
Singh fails to teach:
warping the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
generating, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Nepveu teaches:
warping the first key frame to a second viewpoint to provide a warped first key frame (Nepveu [Pg 21 Par 2] The method includes acquiring a first image of an environment from a first image sensor, the first image sensor being associated with first endogenous parameters; performing a warping operation on the first image according to the viewpoint offset values to generate a warped first image to account for viewpoint differences between the first image sensor and the user of the electronic device;);
Kopp teaches:
generating, by a third diffusion network, a middle frame between the first key frame and the second key frame based on the text description input, the HD map input, and projections from the first key frame and the second key frame (Kopp [0132] A 129.sup.th implementation may further extend the 128.sup.th implementation. In the 129.sup.th implementation, the new frame is generated based on interpolation using two existing frames of the video.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 17.
Singh, Nepveu, and Kopp teach:
The non-transitory computer-readable medium of claim 15, wherein the method further comprises:
generating a trajectory between the first key frame and the second key frame, wherein the middle frame is generated at a point along the trajectory (Kopp [0493] In an example, processing logic may provide an option to keep one jaw/dental arch fixed on the screen, and may only apply a relative movement to the other jaw based on a selected image. This may enable a doctor or patient to focus on a specific jaw for a 3D scene fixed on a screen and observe how the other jaw moves relative to the fixed jaw. For example, processing logic may provide functionality of a virtual articulator model or jaw motion device, where a movement trajectory is dictated by the selected images.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 18.
Singh, Nepveu, and Kopp teach:
The non-transitory computer-readable medium of claim 15, wherein the method further comprises:
applying a warp frame to enforce consistency between the first key frame and the second key frame (Kopp [0324] In embodiments, the modified frame generator 336 performs a warp in the feature space based on the received optical flow (which may also be in the feature space). The modified frame generator 336 may generate modified frames with post-treatment or post-alteration teeth in a manner that reduces flow loss (e.g., perceptual correctness loss in feature space) and/or affine regularization loss for optical flow.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Regarding claim 19.
Singh, Nepveu, and Kopp teach:
The non-transitory computer-readable medium of claim 15,
wherein the first diffusion network, the second diffusion network, and the third diffusion network share weights (Kopp [0335] Training of a neural network may be achieved in a supervised learning manner, which involves feeding a training dataset consisting of labeled inputs through the network, observing its outputs, defining an error (by measuring the difference between the outputs and the label values), and using techniques such as deep gradient descent and backpropagation to tune the weights of the network across all its layers and nodes such that the error is minimized. [0340] The next layer is called a hidden layer, and nodes at the hidden layer each receive one or more of the input values. Each node contains parameters (e.g., weights) to apply to the input values. Each node therefore essentially inputs the input values into a multivariate function (e.g., a non-linear mathematical transformation) to produce an output value. A next layer may be another hidden layer or an output layer. In either case, the nodes at the next layer receive the output values from the nodes at the previous layer, and each node applies weights to those values and then generates its own output value.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh with Nepveu and Kopp. Warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, as in Nepveu and Kopp, would benefit the Singh teachings by allowing for a way to create videos with frames in between them, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, warping images to create warped images, generating different frames in between frames, and creating a new viewpoint, to yield predictable results.
Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al. (US 20240135513) in view of Nepveu et al. (KR 20230116765), Kopp et al. (US 20240185518), and Lei et al. (RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models).
Regarding claim 2.
Singh, Nepveu, and Kopp teach:
The method of claim 1,
Singh, Nepveu, and Kopp fail to teach:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Lei teaches:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Lei. Using RGBD diffusion networks, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to customize the diffusion network, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using RGBD diffusion networks, to yield predictable results.
Regarding claim 9.
Singh, Nepveu, and Kopp teach:
The system of claim 8,
Singh, Nepveu, and Kopp fail to teach:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Lei teaches:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Lei. Using RGBD diffusion networks, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to customize the diffusion network, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using RGBD diffusion networks, to yield predictable results.
Regarding claim 16.
Singh, Nepveu, and Kopp teach:
The non-transitory computer-readable medium of claim 15,
Singh, Nepveu, and Kopp fail to teach:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Lei teaches:
wherein the first diffusion network, the second diffusion network, and the third diffusion network comprise red, green, blue, depth (RGBD) diffusion networks (Lei [Abstract] We practically implement the RGBD inpainting network as a versatile RGBD diffusion model, which is previously used for 2D generative modeling; we make a modification to its reverse diffusion process to enable our use. We evaluate our approach on the task of 3D scene synthesis from sparse RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the superiority of our approach over existing ones.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Lei. Using RGBD diffusion networks, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to customize the diffusion network, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using RGBD diffusion networks, to yield predictable results.
Claim(s) 6, 7, 13, 14, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al. (US 20240135513) in view of Nepveu et al. (KR 20230116765), Kopp et al. (US 20240185518), and Peake et al. (US 20200074266).
Regarding claim 6.
Singh, Nepveu, and Kopp teach:
The method of claim 1,
Singh, Nepveu, and Kopp fail to teach:
further comprising generating the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Peake teaches:
further comprising generating the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Peake. Using something to train an autonomous driving system, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to train different types of systems, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using something to train an autonomous driving system, to yield predictable results.
Regarding claim 7.
Singh, Nepveu, and Kopp teach:
The method of claim 1,
Singh, Nepveu, and Kopp fail to teach:
wherein the simulated scene is employed to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Peake teaches:
wherein the simulated scene is employed to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Peake. Using something to train an autonomous driving system, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to train different types of systems, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using something to train an autonomous driving system, to yield predictable results.
Regarding claim 13.
Singh, Nepveu, and Kopp teach:
The system of claim 8,
Singh, Nepveu, and Kopp fail to teach:
wherein the processor is further configured to: generate the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Peake teaches:
wherein the processor is further configured to: generate the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Peake. Using something to train an autonomous driving system, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to train different types of systems, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using something to train an autonomous driving system, to yield predictable results.
Regarding claim 14.
Singh, Nepveu, and Kopp teach:
The system of claim 8,
Singh, Nepveu, and Kopp fail to teach:
wherein the simulated scene is employed to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Peake teaches:
wherein the simulated scene is employed to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Peake. Using something to train an autonomous driving system, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to train different types of systems, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using something to train an autonomous driving system, to yield predictable results.
Regarding claim 20.
Singh, Nepveu, and Kopp teach:
The non-transitory computer-readable medium of claim 15,
Singh, Nepveu, and Kopp fail to teach:
wherein the method further comprises: generating the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Peake teaches:
further comprising generating the simulated scene from one or more middle frames to train an autonomous driving system (Peake [0034] Accordingly, a software architecture includes an automated training dataset generator that generates feature training datasets based on simulated or virtual environments. The feature training datasets may be used to train various machine learning models for use in real-world autonomous driving applications, e.g., to control the maneuvering of autonomous vehicles. The feature training datasets may include virtual data based on photo-realistic scenes (e.g., simulated 2D image data), depth-map realistic scenes (e.g., simulated 3D image data), and/or environment-object data (e.g., simulated data defining how objects or surfaces interact), each corresponding to the same virtual environment. For example, the environment-object data for a particular vehicle in the virtual environment may relate to the vehicle's motion (e.g., position, velocity, acceleration, trajectory, etc.).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Singh, Nepveu, and Kopp with Peake. Using something to train an autonomous driving system, as in Singh, Nepveu, and Kopp, would benefit the Lei teachings by being able to train different types of systems, this way you can provide multiple viewpoints and create a new video. Additionally, this is the application of a known technique, using something to train an autonomous driving system, to yield predictable results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights