DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This final office action is in response to the amendment filed 23 June 2026.
Claims 1-13, 15-29, and 31-35 are pending. Claims 14 and 30 are cancelled.
Claim Interpretation
The applicant acknowledges that claim 33 invokes 35 USC 112(f) and should be interpreted as such (page 8).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-13, 17-18, 20-29, and 33-35 are rejected under 35 U.S.C. 103 as being unpatentable over Filmonov (WO2023/277722, published 5 January 2023, provided by application on IDS filed 25 February 2025) and further in view of Fischer et al. (US 2025/0076874, filed 31 August 2023, hereafter Fischer) and further in view of Thakur et al. (PReLU and edge-aware filter-based image denoiser using convolutional neural networks, published 16 February 2021, hereafter Thakur).
As per independent claim 1, Filmonov discloses a system comprising:
execute a machine learning model on the image data, the machine learning model comprising a plurality of layers (claim 1: Here, a multimodal neural network model (machine learning model) is applied to image data to encode the image, decode the depth, and perform semantic segmentation to determine semantic labels for the image)
applying a non-linear mapping function to output of one layer of the plurality of layers to generate depth data (claim 2: Here, a first layer and third layer are operable to perform convolutions, normalizations, and non-linearity functions on the image data (paragraphs 0026-0028))
train the machine learning model based on the depth data to generate a trained machine learning model (Figure 7; paragraphs 0039-0044: Here, the neural network model is trained)
Filmonov fails to specifically disclose:
memory configured to store image data captured by a plurality of camera
one or more processors communicatively coupled to the memory
wherein the output of the one layer does not include disparity data and train the machine learning model without determining disparity data
However, Fischer, which is analogous to the claimed invention because it is directed toward an autonomous driving system, discloses:
memory configured to store image data captured by a plurality of camera (paragraph 0005: Here, a memory receives a video stream (image data) from a camera of the vehicle. Further, the vehicle may include a plurality of cameras each associated with a respective video stream (paragraph 0075))
one or more processors communicatively coupled to the memory, the one or more processors being configured to execute functions (paragraph 0005: Here, the processor is configured to execute instructions stored in the memory)
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Fischer with Filmonov, with a reasonable expectation of success, as it would have allowed for implementation of an apparatus for executing performance of autonomous vehicle navigation (Fischer: paragraph 0005).
However, Thakur, which is analogous to the claimed invention because it is directed toward using PreLU activation functions, discloses wherein the output of the one layer does not include disparity data and train the machine learning model without determining disparity data (Section 4: Here, the feed forward CNN is designed based on residual learning. In this instance, the residual image is obtained as output and a loss function is formulated using residual mapping designed with CNN parameters. A normalization or scaling and shifting an operator before each non-linear activation unit is performed and along with PReLU activation, the parameters are updated during back propagation). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Thakur with Filmonov-Fischer, with a reasonable expectation of success, as it would have allowed improved convergence of CNN training (Thakur: Section 4).
As per dependent claim 2, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses at least one of a loss function, an input depth of the non-linear mapping function, or a probability distribution of an output depth of the non-linear mapping function (paragraph 0027: Here, a disparity (loss) is calculated for the non-linear mapping functions). However, Filmonov fails to specifically disclose a slope of the mapping function.
However, Thakur, which is analogous to the claimed invention because it is directed toward finding a proper slope, discloses a slope of the mapping function. (Section 1: Here, a CNN activation function used at each layer may be implemented using ReLU, a sigmoid function, or PReLU. ReLu produces zero output for negative values while sigmoid functions result in a gradient that tends toward zero. However, use of PReLU results in finding a proper slope for both positive and negative values). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Thakur with Filmonov-Fisher, with a reasonable expectation of success, as it would have allowed for improving performance of the neural network by considering both positive and negative parts of the non-linear activation function (Thakur: Section 1).
As per dependent claim 4, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses operation of a device based on the trained machine learning model (paragraphs 0002-0003: Here, an autonomous vehicle is controlled via the Advanced Driver Assistance Systems (ADAS)).
As per dependent claim 5, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 4, and the same rejection is incorporated herein. Filmonov discloses wherein the device comprises a vehicle or a robot and wherein as part of controlling operation of the vehicle or robot… navigate the vehicle or the robot in the environment (paragraphs 0002-0003 and 0018: Here, a neural network model is used for controlling vehicles using the ADAS).
As per dependent claim 6, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Fischer discloses the plurality of cameras comprising at least three cameras, the at least three cameras being configured to capture the image data, and each of the at least three cameras having a different field of view (paragraph 0075: Here, a first camera is a front facing camera and captures image data in front of the vehicle. The second camera is a left facing camera and captures image data to the left of the vehicle. The third camera is a right facing camera and captures image data to the right of the camera).
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Fischer with Filmonov, with a reasonable expectation of success, as it would have allowed for capturing image data from a plurality of fields of view in order to provide multiple views to improve situational awareness (Fischer: paragraph 0075).
As per dependent claim 7, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses wherein the one layer is an output layer and wherein as part of applying the non-linear mapping function, apply the non-linear mapping function to the output of the machine learning model (Figure 1-3; paragraphs 0018-0028: Here, output from one layer may have the non-linear mapping function applied).
As per dependent claim 8, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses wherein the one layer is a middle layer and wherein as part of applying the non-linear mapping function, apply the non-linear mapping function in an output layer of the machine learning model (Figure 1-3; paragraphs 0018-0028: Here, output from one layer may have the non-linear mapping function applied).
As per dependent claim 9, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses wherein the plurality of layers comprises at least one activation function layer configured to apply an activation function to an output of a respective previous layer (Figure 2, item 206; Figure 3, item 304; paragraphs 0023 and 0027: Here, ReLu and sigmoid activation functions are applied).
As per dependent claim 10, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 9, and the same rejection is incorporated herein. Filmonov discloses wherein the at least one activation function layer comprises at least one middle layer, and wherein the activation function of the at least one middle layer comprises at least one of rectified linear unit (ReLU), parametric rectified linear unit (PReLU), or exponential linear unit (ELU) (Figure 2, item 206; Figure 3, item 304; paragraphs 0023 and 0027: Here, ReLu and sigmoid activation functions are applied).
As per dependent claim 11, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 10, and the same rejection is incorporated herein. Filmonov fails to specifically disclose PReLU.
However, Thakur, which is analogous to the claimed invention because it is directed toward a PReLU in a convolutional neural networks, discloses PReLU (Section 3). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Thakur with Filmonov-Fischer, with a reasonable expectation of success, as it would have allowed for substituting ReLU with PReLU in order to achieve adaptive activation during training.
As per dependent claim 12, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 11, and the same rejection is incorporated herein. Filmonov discloses wherein the at least one activation function layer comprises at least one output layer, and wherein the activation function of the at least one output layer comprises at least one of rectified linear unit (ReLU), parametric rectified linear unit (PReLU), or exponential linear unit (ELU) (Figure 2, item 206; Figure 3, item 304; paragraphs 0023 and 0027: Here, ReLu and sigmoid activation functions are applied).
As per dependent claim 13, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 12, and the same rejection is incorporated herein. Filmonov fails to specifically disclose PReLU.
However, Thakur, which is analogous to the claimed invention because it is directed toward a PReLU in a convolutional neural networks, discloses PReLU (Section 3). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Thakur with Filmonov-Fischer, with a reasonable expectation of success, as it would have allowed for substituting ReLU with PReLU in order to achieve adaptive activation during training.
With respect to claim 17, the claim recites the limitations substantially similar to those in claim 1. The analysis of claim 1 incorporated herein and claim 17 is rejected under similar rationale.
With respect to claim 18, the claim recites the limitations substantially similar to those in claim 2. The analysis of claim 2 incorporated herein and claim 18 is rejected under similar rationale.
With respect to claims 20-26, the claims recite the limitations substantially similar to those in claims 4-10, respectively. The analysis of claims 4-10 incorporated herein and claims 20-26 are rejected under similar rationale.
With respect to claims 27-29, the claim recites the limitations substantially similar to those in claim 11-13, respectively. The analysis of claims 11-13 are incorporated herein and claims 27-29 are rejected under similar rationale.
With respect to claim 33, the claim recites the limitations substantially similar to those in claim 1. The analysis of claim 1 incorporated herein and claim 33 is rejected under similar rationale.
With respect to claim 34, the claim recites the limitations substantially similar to those in claim 1. The analysis of claim 1 incorporated herein and claim 34 is rejected under similar rationale.
With respect to claim 35, the claim recites the limitations substantially similar to those in claim 4. The analysis of claim 4 incorporated herein and claim 35 is rejected under similar rationale.
Claims 3 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Filmonov, Fischer, and Thakur, and further in view of Vogel et al. (US 2024/0412032, filed 9 June 2023, hereafter Vogel).
As per dependent claim 3, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 2, and the same rejection is incorporated herein. Filmonov discloses wherein the slope of the non-linear mapping function at least one of a) increases with the input depth of the non-linear mapping function or b) decreases with a higher probability density of the output depth of the non-linear mapping function (paragraph 0027: Here, the activation function for the depth decoder is a sigmoid function. Based on the provided function, slope of the function increases as the predicted depth increases).
Filmonov fails to specifically disclose a mean absolute relative error loss function. However, Vogel, which is analogous to the claimed invention because it is directed toward artificial neural networks, discloses a mean absolute relative error loss function (paragraph 0026). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Vogel with Filmonov-Fischer-Thakur, with a reasonable expectation of success, as it would have allowed for evaluating an architecture (Vogel: paragraph 0026).
With respect to claim 19, the claim recites the limitations substantially similar to those in claim 3. The analysis of claim 3 incorporated herein and claim 19 is rejected under similar rationale.
Claims 15-16 and 31-32 are rejected under 35 U.S.C. 103 as being unpatentable over Filmonov, Fischer, and Thakur and further in view of Gazvoda (US 2025/0080425, filed 1 September 2023) and further in view of Alroobaea et al. (US 2023/0308465, filed 12 April 2023, hereafter Alroobaea).
As per dependent claim 15, Filmonov, Fischer, and Thakur disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Filmonov discloses wherein the non-linear mapping function is a first non-linear mapping function and wherein the depth data is first depth data (claim 2: Here, a first layer and third layer are operable to perform convolutions, normalizations, and non-linearity functions on the image data (paragraphs 0026-0028)).
Filmonov fails to specifically disclose:
determine a change in an environment
based on the change in the environment, apply a second non-linear mapping function of the plurality of non-linear mapping functions to the output of the one layer of the plurality of layers to generate second depth data
further train the trained machine learning model based on the second depth data
However, Gazvoda, which is analogous to the claimed invention because it is directed toward reinforcement learning, discloses:
determine a change in an environment (paragraphs 0053-0054: Here, a training environment has a plurality of states, and it is determined that the environment is moving from a first state to a second state)
further train the trained machine learning model based on the second depth data (paragraphs 0053-0054: Here, a machine learning model is trained based upon the updated state)
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Gazvoda with Filmonov-Fischer-Thakur, with a reasonable expectation of success, as it would have allowed for generating an updated training model based upon the current environmental state (Gazvoda: paragraph 0054).
Additionally, Alroobaea, which is analogous to the claimed invention because it is directed toward selecting a non-linear mapping function, discloses:
apply a second non-linear mapping function of the plurality of non-linear mapping functions to the output of the one layer of the plurality of layers to generate second depth data (paragraphs 0093 and 0122: Here, an appropriate activation function is selected from amongst a plurality of non-linear mapping functions, such as ReLU or sigmoid functions. Additionally, based on the parameters of the dynamic neural network, the depth may be modified). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Alroobaea with Filmonov-Fischer-Gazvoda, with a reasonable expectation of success, as it would have allowed for optimizing the DNN using different activation functions and depths (Alroobaea: paragraph 0093).
As per dependent claim 16, Filmonov, Fischer, Thakur, Gazvoda, and Alroobaea disclose the limitations similar to those in claim 15, and the same rejection is incorporated herein. Gazvoda discloses as part of the training of the model based upon the environmental change, using the updated trained machine learning model and no longer using the previous model (paragraph 0054). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Gazvoda with Filmonov-Fischer-Thakur, with a reasonable expectation of success, as it would have allowed for generating an updated training model based upon the current environmental state (Gazvoda: paragraph 0054).
Additionally, Alroobaea, which is analogous to the claimed invention because it is directed toward selecting a non-linear mapping function, discloses:
apply a second non-linear mapping function of the plurality of non-linear mapping functions to the output of the one layer of the plurality of layers to generate second depth data (paragraphs 0093 and 0122: Here, an appropriate activation function is selected from amongst a plurality of non-linear mapping functions, such as ReLU or sigmoid functions. Additionally, based on the parameters of the dynamic neural network, the depth may be modified). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Alroobaea with Filmonov-Fischer-Gazvoda, with a reasonable expectation of success, as it would have allowed for optimizing the DNN using different activation functions and depths (Alroobaea: paragraph 0093).
With respect to claims 31-32, the claim recites the limitations substantially similar to those in claim 15-16, respectively. The analysis of claim 15-16 incorporated herein and claims 31-32 are rejected under similar rationale.
Response to Arguments
Applicant’s arguments with respect to the rejection of claims under 35 USC 103 in view of Filimonov and Fisher have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Filimonov, Fisher, and Thakur.
As noted by the applicant, the amended claim limitations incorporate the limitations of dependent claim 14 into independent claim 1 (page 9). The applicant argues that Thakur does not disclose disparity data because Thakur does not process depth or disparity data (page 10).
However, the claimed invention recites “wherein the output of the one layer does not include disparity data (claim 1, lines 8-9)” and “generate a trained machine learning model without determining disparity data (claim 1, lines 10-11).” Thakur, which is analogous to the claimed invention because it is directed toward using PreLU activation functions, discloses wherein the output of the one layer does not include disparity data and train the machine learning model without determining disparity data (Section 4). Specifically, the feed forward CNN is designed based on residual learning. In this instance, the residual image is obtained as output and a loss function is formulated using residual mapping designed with CNN parameters. A normalization or scaling and shifting an operator before each non-linear activation unit is performed and along with PReLU activation, the parameters are updated during back propagation.
The applicant further argues that modifying Filimonov’s depth estimation network to remove the disparity calculation based on Thakur’s image denoiser would change the principle operation of the prior art invention (page 10). The examiner respectfully disagrees. While Filimonov discloses a disparity calculation, Thakur discloses training the machine learning model using residual learning. It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to replace the disparity calculation of Filimonov with Thakur’s residual learning using a loss function, with a reasonable expectation of success, as it would have allowed improved convergence of CNN training (Thakur: Section 4).
For these reasons, these arguments are not persuasive.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Xiong et al. (US 2024/0223739): Discloses training a model without using disparity information to simply and reducing training costs associated with training (paragraph 0066)
Venkataraman et al. (US 2022/0414928): Discloses models trained using depth from disparity include noisy or inconsistent estimates (paragraph 0194)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE R STORK whose telephone number is (571)272-4130. The examiner can normally be reached 8am - 2pm; 4pm - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571/272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE R STORK/Primary Examiner, Art Unit 2128
Prosecution Insights