DETAIL OFFICE ACTIONS
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 04/02/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Amendment
Applicant submitted amendments on 04/02/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Applicant Arguments:
In regards to Argument 1, Applicant/s state/s that the cited prior arts do not teach the amended claims 1, 8 and 15. Specially, the limitation “comparing disparity estimations, which are images of height by width where each pixel is a disparity value, from the stereo architecture with ground truth disparity from the graphic rendering system to generate training feedback”; therefore, the rejection under 35 U.S.C. 103 should be withdrawn.
In regards to Argument 2, Applicant/s state/s that the cited prior arts do not teach the amended claims 5, 12 and 19. Specially, limitation “the one or more 3D convolution networks are configured to learn the disparity estimation at a predetermined resolution less than a resolution of the plurality of stereo image pairs”; therefore, the rejection under 35 U.S.C. 103 should be withdrawn.
Examiner’s Responses:
PNG
media_image1.png
823
1872
media_image1.png
Greyscale
In response to Argument 1, The Examiner respectfully disagrees. The Examiner states that Duggal teaches a system for estimating depth from two stereo images. Specifically, the system uses an end-to-end differentiable machine-learning model that generates output including sparse disparity estimates associated with disparities between the pair of stereo images. The estimated sparse disparity then be refined and utilized to build a disparity map; the system then converts the disparity map into three-dimensional depth estimates. In addition, Duggal teaches training the machine-learning model by using a loss function. The loss function includes the difference between estimated disparity map of a training image and a ground-truth disparity map (Duggal, ¶ [0048-0049], see reconstructed text below).
The applicant is making arguments that the limitation “comparing disparity estimations, which are images of height by width where each pixel is a disparity value, from the stereo architecture with ground truth disparity from the graphic rendering system to generate training feedback” is not met; however, the Examiner respectfully disagrees. The Applicant is making argument that a definition of “cost disparity” is absent in Duggal; however, Duggal clearly teaches the “cost disparity” and “ground truth disparity” are estimated from respective disparity maps, as disclosed above in annotated paragraph ¶ 0048-0049. The Examiner finds that the limitations of the claims were identified and correlated with the references as indicated above. Therefore, the Examiner will maintain the rejection.
In response to Argument 2, The Examiner respectfully disagrees. The Examiner finds that Duggal teaches extracting corresponding feature maps from the stereo images, then performing a matching algorithm to extracted feature maps and generate output including sparse disparity estimates associated with disparities between the pair of stereo images. In addition, Duggal teaches downscaling the extracted feature map (e.g., to one quarter of the original input image. Thus, the estimated disparities are learned from feature maps of the stereo image pairs, and the feature maps’ resolution is less than resolution of the stereo image pairs by a predetermined factor (one quarter). Therefore, the limitation “the one or more 3D convolution networks are configured to learn the disparity estimation at a predetermined resolution less than a resolution of the plurality of stereo image pairs” reads on Duggal’s teaching. Counsel's assertion that Duggal’s learning disparity estimation from downscaled feature maps of the stereo image pair does not meet the claim limitation is merely an argument unaccompanied by evidentiary support, and, thus, is insufficient to rebut Examiner's finding of obviousness. Arguments of counsel cannot take the place of evidence in the record. In re Schulze, 346 F.2d 600, 602, 145 USPQ 716, 718 (CCPA 1965); In re Geisler, 116 F.3d 1465, 43 USPQ2d 1362 (Fed. Cir. 1997) (“An assertion of what seems to follow from common experience is just attorney argument and not the kind of factual evidence that is required to rebut a prima facie case of obviousness.”). MPEP §§ 2145, 2129, 2144.03, 716.01(c). To overcome such an interpretation and establish a particular meaning (that is not already a special definition in the specification), Applicant is required to provide reliable extrinsic evidence such as a reputable optical journal or text showing the asserted meaning and usage. (See, e.g., Ex Parte Longo Appeal 2009014183; Appl. No. 11/328,537).
Last, The Examiner made a proper determination of obviousness under 35 U.S.C. §103, and also provided an appropriate supporting rationale in view of the recent decision by the Supreme Court in KSR International Co. v. Teleflex Inc. (KSR), 550 U.S. 398, 82 USPQ2d 1385 (2007). The Examiner’s rational are based on the Office’s current understanding of the law, and are believed to be fully consistent with the binding precedent of the Supreme Court. Furthermore, the Examiner supported the rejection under 35 U.S.C. §103 via making the clear articulation of the reason(s) why the claimed invention would have been obvious by citing the specific areas in the prior art references. Further the Examiner, clearly stating the modification of the inventions, supported the rejection under 35 U.S.C. §103 by making the analysis explicit. Last, the Examiner did not make conclusory statements. The Court quoting In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006), stated that “‘[R]ejections on obviousness cannot be sustained by mere conclusory statements; instead, there must be some articulated reasoning with some rational underpinning to support the legal conclusion of obviousness.’” KSR, 550 U.S. at ___, 82 USPQ2d at 1396. Therefore, the Examiner has established a proper 35 U.S.C. §103 rejection with Duggal in view of Reyes, which is disclosed in detail below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-6, 8-9, 11-13, 15-16 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Duggal et al. (US-20200302627-A1, hereinafter Duggal) in view of Reyes et al. (Reyes, M. Fuentes, Pablo d’Angelo, and Friedrich Fraundorfer. "An evaluation of stereo and multiview algorithms for 3d reconstruction with synthetic data." The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 48 (2023), hereinafter Reyes).
CLAIM 1
In regards to Claim 1, Duggal teaches a method for training (Duggal, ¶ [0155]: “a machine learning computing system ”; ¶ [0047-0048]: “Training the machine-learned models can include training the machine-learned models in the pipeline end-to-end”) a learned stereo architecture. (Duggal, ¶ [0024 and 0055]: “the depth estimation computing system can use machine-learned models to determine the sparse disparity estimates by estimating the disparities between the pair of stereo images based on comparisons of respective portions of the pair of stereo images. The machine-learned models can also later be used in the determination of confidence ranges and a disparity map for the pair of stereo images…”, see FIG. 2. Duggal discloses a depth estimation system that take stereo images as input and output disparity and depth information)
Duggal does not explicitly disclose receiving, from a graphic rendering system, a plurality of stereo image pairs comprising a variety of disparate scenes and scene parameters, wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline;
Reyes is in the same field of art of disparity estimation using stereo networks. Further, Reyes teaches receiving, from a graphic rendering system (Reyes, page 1022, section 2.3: “…synthetic data can be generated with thousands of samples and accurate ground truth, as the geometric details of the 3D models can be retrieved by the rendering software”. The Examiner notes graphic rendering software implies a computer system with graphic processing unit (GPU)), a plurality of stereo image pairs (Reyes, page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks … By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”) comprising a variety of disparate scenes and scene parameters (Reyes, page 1023, section 3.1.2 SyntCities preparation: “Three 3D city models are used to render the dataset: Paris, Venice and New York. The samples are given for ground sample distances (GSD) of 10cm, 30 cm and 100 cm and provided with training and testing subsets… the camera parameters are available” The Examiner notes GSD and camera parameters correspond to “scene parameter”), wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, there are 6 images/views for each scene, image pair (3)-(4)
PNG
media_image2.png
907
923
media_image2.png
Greyscale
has a baseline 1, pair (2)-(5) has a baseline 2 and pair (1)-(6) has a baseline 3);
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Duggal by incorporating the synthetic SyntCities dataset that is taught by Reyes, to make a stereo disparity estimation network that is trained with a multiple baselines dataset; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to use synthetic dataset for larger amount of samples and more accurate ground truth (Reyes, page 1022, section 2.3: “datasets are regularly not enough to train a neural network model because of their size and the incomplete ground truth. To help overcome this, synthetic data can be generated with thousands of samples and accurate ground truth”).
The combination of Duggal and Reyes teaches inputting the plurality of stereo image pairs (Duggal, ¶ [0030]: “the one or more machine-learned models can be configured to receive input including the pair of stereo images”) into a stereo comprising one or more 3D convolution networks (Duggal, ¶ [0103-0105]: “The confidence range prediction operations 206 can include the use of one or more machine-learned models … a convolutional encoder-decoder structure ... the cost aggregation operations can include three-dimensional cost volume estimation and spatial aggregation … generating output including the cost over the disparity range at the size B×R×H×W, in which B represents the matching window feature size, R represents the number of disparities per pixel, H represents the height of an image, and W represents the width of an image”; see modified FIG. 2 below, Duggal teaches a convolution network that performs 3D cost volume estimation and spatial aggregation. The Examiner interprets “3D convolution network” as convolutional neural network that operate over height × width × depth data)
PNG
media_image3.png
597
545
media_image3.png
Greyscale
configured to learn disparity estimation based on the plurality of stereo image pairs architecture (Duggal, ¶ [0024 and 0055]: “the depth estimation computing system can use machine-learned models to determine the sparse disparity estimates by estimating the disparities between the pair of stereo images based on comparisons of respective portions of the pair of stereo images. The machine-learned models can also later be used in the determination of confidence ranges and a disparity map for the pair of stereo images…”, see FIG. 2. Duggal discloses a depth estimation system that take stereo images as input and output disparity and depth information);
comparing disparity estimations, which are images of height by width where each pixel is a disparity value, from the stereo architecture with ground truth disparity from the graphic rendering system to generate training feedback (Duggal, ¶ [0048-0049], see reconstructed text with annotation below. Duggal teaches training the model using a loss function; the loss function includes difference between estimated disparity map and ground truth disparity map);
PNG
media_image1.png
823
1872
media_image1.png
Greyscale
and adjusting one or more neural network models implemented by the stereo architecture based on the training feedback thereby configuring the learned stereo architecture. (Duggal, ¶ [0173-0175]: “training the one or more machine-learned models 1110 and/or the one or more machine-learned models 1140 can include the use of backpropagation to learn parameters”. Duggal discloses training ML models using backpropagation, i.e. adjusting models’ parameter to minimize a loss function)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 2
In regards to Claim 2, the combination of Duggal and Reyes teaches the method of Claim 1. In addition, the combination of Duggal and Reyes teaches the plurality of stereo image pairs utilized to train the stereo architecture are fully synthetic image data. (Reyes, page 1021, right col, last paragraph: “We prepared synthetic data to be compatible with stereo and MVS frame works”; page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks with patches resembling remote sensing scenes and under controlled simulated conditions. Three 3D city models are used to render the dataset: Paris, Venice and New York”. SyntCities is a synthetic dataset that are rendered from 3D city models.)
CLAIM 4
In regards to Claim 4, the combination of Duggal and Reyes teaches the method of Claim 1. In addition, the combination of Duggal and Reyes teaches the first baseline corresponds to a first stereo image system and the second baseline corresponds to a second stereo image system. (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, each baseline corresponds to different simulated camera pair)
PNG
media_image2.png
907
923
media_image2.png
Greyscale
CLAIM 5
In regards to Claim 5, the combination of Duggal and Reyes teaches the method of Claim 1. In addition, the combination of Duggal and Reyes teaches the one or more 3D convolution networks are configured to learn the disparity estimation at a predetermined resolution less than a resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”)
CLAIM 6
In regards to Claim 6, the combination of Duggal and Reyes teaches the method of Claim 1. In addition, the combination of Duggal and Reyes teaches the predetermined resolution is at least one of a factor of 2, 4, or 8 less than the resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”
***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
CLAIM 8
In regards to Claim 8, Duggal teaches an apparatus for training a learned stereo architecture (Duggal, ¶ [0155]: “a machine learning computing system”; Abstract: “A disparity map for the stereo images can be generated based on using the confidence ranges and machine-learned models”. Duggal teaches a system to train ML model that can estimate disparity map from stereo images), comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions (Duggal, ¶ [0163-0165]: “The machine learning computing system includes one or more processors and a memory … The memory can also store computer-readable instructions that can be executed by the one or more processors”)
Duggal does not explicitly disclose receiving, from a graphic rendering system, a plurality of stereo image pairs comprising a variety of disparate scenes and scene parameters, wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline;
Reyes is in the same field of art of disparity estimation using stereo networks. Further, Reyes teaches receiving, from a graphic rendering system (Reyes, page 1022, section 2.3: “…synthetic data can be generated with thousands of samples and accurate ground truth, as the geometric details of the 3D models can be retrieved by the rendering software”. The Examiner notes graphic rendering software implies a computer system with graphic processing unit (GPU)), a plurality of stereo image pairs (Reyes, page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks … By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”) comprising a variety of disparate scenes and scene parameters (Reyes, page 1023, section 3.1.2 SyntCities preparation: “Three 3D city models are used to render the dataset: Paris, Venice and New York. The samples are given for ground sample distances (GSD) of 10cm, 30 cm and 100 cm and provided with training and testing subsets… the camera parameters are available” The Examiner notes GSD and camera parameters correspond to “scene parameter”), wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, there are 6 images/views for each scene, image pair (3)-(4) has a baseline 1, pair (2)-(5) has a baseline 2 and pair (1)-(6) has a baseline 3);
PNG
media_image2.png
907
923
media_image2.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Duggal by incorporating the synthetic SyntCities dataset that is taught by Reyes, to make a stereo disparity estimation network that is trained with a multiple baselines dataset; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to use synthetic dataset for larger amount of samples and more accurate ground truth (Reyes, page 1022, section 2.3: “datasets are regularly not enough to train a neural network model because of their size and the incomplete ground truth. To help overcome this, synthetic data can be generated with thousands of samples and accurate ground truth”).
The combination of Duggal and Reyes teaches inputting the plurality of stereo image pairs (Duggal, ¶ [0030]: “the one or more machine-learned models can be configured to receive input including the pair of stereo images”) into a stereo comprising one or more 3D convolution networks (Duggal, ¶ [0103-0105]: “The confidence range prediction operations 206 can include the use of one or more machine-learned models … a convolutional encoder-decoder structure ... the cost aggregation operations can include three-dimensional cost volume estimation and spatial aggregation … generating output including the cost over the disparity range at the size B×R×H×W, in which B represents the matching window feature size, R represents the number of disparities per pixel, H represents the height of an image, and W represents the width of an image”; see modified FIG. 2 below, Duggal teaches a convolution network that performs 3D cost volume estimation and spatial aggregation. The Examiner interprets “3D convolution network” as convolutional neural network that operate
PNG
media_image3.png
597
545
media_image3.png
Greyscale
over height × width × depth data)
configured to learn disparity estimation based on the plurality of stereo image pairs architecture (Duggal, ¶ [0024 and 0055]: “the depth estimation computing system can use machine-learned models to determine the sparse disparity estimates by estimating the disparities between the pair of stereo images based on comparisons of respective portions of the pair of stereo images. The machine-learned models can also later be used in the determination of confidence ranges and a disparity map for the pair of stereo images…”, see FIG. 2. Duggal discloses a depth estimation system that take stereo images as input and output disparity and depth information);
PNG
media_image1.png
823
1872
media_image1.png
Greyscale
comparing disparity estimations, which are images of height by width where each pixel is a disparity value, from the stereo architecture with ground truth disparity from the graphic rendering system to generate training feedback (Duggal, ¶ [0048-0049], see reconstructed text with annotation below. Duggal teaches training the model using a loss function; the loss function includes difference between estimated disparity map and ground truth disparity map);
and adjusting one or more neural network models implemented by the stereo architecture based on the training feedback thereby configuring the learned stereo architecture. (Duggal, ¶ [0173-0175]: “training the one or more machine-learned models 1110 and/or the one or more machine-learned models 1140 can include the use of backpropagation to learn parameters”. Duggal discloses training ML models using backpropagation, i.e. adjusting models’ parameter to minimize a loss function)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 9
In regards to Claim 9, the combination of Duggal and Reyes teaches the apparatus of Claim 8. In addition, the combination of Duggal and Reyes teaches the plurality of stereo image pairs utilized to train the stereo architecture are fully synthetic image data. (Reyes, page 1021, right col, last paragraph: “We prepared synthetic data to be compatible with stereo and MVS frame works”; page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks with patches resembling remote sensing scenes and under controlled simulated conditions. Three 3D city models are used to render the dataset: Paris, Venice and New York”. SyntCities is a synthetic dataset that are rendered from 3D city models.)
CLAIM 11
In regards to Claim 11, the combination of Duggal and Reyes teaches the apparatus of Claim 8. In addition, the combination of Duggal and Reyes teaches the first baseline corresponds to a first stereo image system and the second baseline corresponds to a second stereo image system. (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, each baseline corresponds to different simulated camera pair)
PNG
media_image2.png
907
923
media_image2.png
Greyscale
CLAIM 12
In regards to Claim 12, the combination of Duggal and Reyes teaches the apparatus of Claim 8. In addition, the combination of Duggal and Reyes teaches the one or more 3D convolution networks are configured to learn the disparity estimation at a predetermined resolution less than a resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”)
CLAIM 13
In regards to Claim 13, the combination of Duggal and Reyes teaches the apparatus of Claim 12. In addition, the combination of Duggal and Reyes teaches the predetermined resolution is at least one of a factor of 2, 4, or 8 less than the resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”
***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
CLAIM 15
In regards to Claim 15, Duggal teaches a non-transitory computer-readable medium comprising processor-executable instructions (Duggal, ¶ [0158-0159]: “the memory (e.g., one or more non-transitory computer-readable storage mediums…)”) that, when executed by one or more processors (Duggal, ¶ [0158-0159]: “The memory can also store computer-readable instructions”) of an apparatus (Duggal, ¶ [0157-0160]: “The computing system … generating a disparity map for the pair of stereo images …”. Duggal teaches a system to train ML model that can estimate disparity map from stereo images)
Duggal does not explicitly disclose receiving, from a graphic rendering system, a plurality of stereo image pairs comprising a variety of disparate scenes and scene parameters, wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline;
Reyes is in the same field of art of disparity estimation using stereo networks. Further, Reyes teaches receiving, from a graphic rendering system (Reyes, page 1022, section 2.3: “…synthetic data can be generated with thousands of samples and accurate ground truth, as the geometric details of the 3D models can be retrieved by the rendering software”. The Examiner notes graphic rendering software implies a computer system with graphic processing unit (GPU)), a plurality of stereo image pairs (Reyes, page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks … By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”) comprising a variety of disparate scenes and scene parameters (Reyes, page 1023, section 3.1.2 SyntCities preparation: “Three 3D city models are used to render the dataset: Paris, Venice and New York. The samples are given for ground sample distances (GSD) of 10cm, 30 cm and 100 cm and provided with training and testing subsets… the camera parameters are available” The Examiner notes GSD and camera parameters correspond to “scene parameter”), wherein: a first subset of stereo image pairs correspond to a first baseline, and a second subset of stereo image pairs correspond to a second baseline different from the first baseline (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, there are 6 images/views for each scene, image pair (3)-(4) has a baseline 1, pair (2)-(5) has a baseline 2 and pair (1)-(6) has a baseline 3);
PNG
media_image2.png
907
923
media_image2.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Duggal by incorporating the synthetic SyntCities dataset that is taught by Reyes, to make a stereo disparity estimation network that is trained with a multiple baselines dataset; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to use synthetic dataset for larger amount of samples and more accurate ground truth (Reyes, page 1022, section 2.3: “datasets are regularly not enough to train a neural network model because of their size and the incomplete ground truth. To help overcome this, synthetic data can be generated with thousands of samples and accurate ground truth”).
The combination of Duggal and Reyes teaches inputting the plurality of stereo image pairs (Duggal, ¶ [0030]: “the one or more machine-learned models can be configured to receive input including the pair of stereo images”) into a stereo comprising one or more 3D convolution networks (Duggal, ¶ [0103-0105]: “The confidence range prediction operations 206 can include the use of one or more machine-learned models … a convolutional encoder-decoder structure ... the cost aggregation operations can include three-dimensional cost volume estimation and spatial aggregation … generating output including the cost over the disparity range at the size B×R×H×W, in which B represents the matching window feature size, R represents the number of disparities per pixel, H represents the height of an image, and W represents the width of an image”; see modified FIG. 2 below, Duggal teaches a convolution network that performs 3D cost volume estimation and spatial aggregation. The Examiner interprets “3D convolution network” as convolutional neural network that operate
PNG
media_image3.png
597
545
media_image3.png
Greyscale
over height × width × depth data)
configured to learn disparity estimation based on the plurality of stereo image pairs architecture (Duggal, ¶ [0024 and 0055]: “the depth estimation computing system can use machine-learned models to determine the sparse disparity estimates by estimating the disparities between the pair of stereo images based on comparisons of respective portions of the pair of stereo images. The machine-learned models can also later be used in the determination of confidence ranges and a disparity map for the pair of stereo images…”, see FIG. 2. Duggal discloses a depth estimation system that take stereo images as input and output disparity and depth information);
PNG
media_image1.png
823
1872
media_image1.png
Greyscale
comparing disparity estimations, which are images of height by width where each pixel is a disparity value, from the stereo architecture with ground truth disparity from the graphic rendering system to generate training feedback (Duggal, ¶ [0048-0049], see reconstructed text with annotation below. Duggal teaches training the model using a loss function; the loss function includes difference between estimated disparity map and ground truth disparity map);
and adjusting one or more neural network models implemented by the stereo architecture based on the training feedback thereby configuring the learned stereo architecture. (Duggal, ¶ [0173-0175]: “training the one or more machine-learned models 1110 and/or the one or more machine-learned models 1140 can include the use of backpropagation to learn parameters”. Duggal discloses training ML models using backpropagation, i.e. adjusting models’ parameter to minimize a loss function)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 16
In regards to Claim 16, the combination of Duggal and Reyes teaches the medium of Claim 15. In addition, the combination of Duggal and Reyes teaches the plurality of stereo image pairs utilized to train the stereo architecture are fully synthetic image data. (Reyes, page 1021, right col, last paragraph: “We prepared synthetic data to be compatible with stereo and MVS frame works”; page 1023, section 3.1.2 SyntCities preparation: “SyntCities is a dataset to train stereo matching networks with patches resembling remote sensing scenes and under controlled simulated conditions. Three 3D city models are used to render the dataset: Paris, Venice and New York”. SyntCities is a synthetic dataset that are rendered from 3D city models.)
CLAIM 18
PNG
media_image2.png
907
923
media_image2.png
Greyscale
In regards to Claim 18, the combination of Duggal and Reyes teaches the medium of Claim 15. In addition, the combination of Duggal and Reyes teaches the first baseline corresponds to a first stereo image system and the second baseline corresponds to a second stereo image system. (Reyes, page 1023, section 3.1.2 SyntCities preparation: “By default, SyntCities images are given in pairs, which are represented for simplicity by the legends Baseline 1, Baseline 2 and Baseline 3 in Figure1”; see FIG. 1 with annotations below, each baseline corresponds to different simulated camera pair)
CLAIM 19
In regards to Claim 19, the combination of Duggal and Reyes teaches the medium of Claim 15. In addition, the combination of Duggal and Reyes teaches the one or more 3D convolution networks are configured to learn the disparity estimation at a predetermined resolution less than a resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”)
CLAIM 20
In regards to Claim 20, the combination of Duggal and Reyes teaches the medium of Claim 19. In addition, the combination of Duggal and Reyes teaches the predetermined resolution is at least one of a factor of 2, 4, or 8 less than the resolution of the plurality of stereo image pairs. (Duggal, ¶ [0097]: “The size of the final feature map generated as part of the feature extraction operations can be smaller than the original input image size (e.g., one quarter of the original input image size)”
***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
Claim(s) 3, 10 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Duggal in view of Reyes, and further in view of D’Angelo et al. (D'Angelo, Pablo et al. "Syntcities: A large synthetic remote sensing dataset for disparity estimation." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 15 (Year: 2022), hereinafter D’Angelo).
CLAIM 3
In regards to Claim 3, the combination of Duggal and Reyes teaches the method of Claim 1.
The combination of Duggal and Reyes does not explicitly disclose the scene parameters comprise at least one of a lighting level, a resolution, a material, a texture, or a surface type.
D’Angelo is in the same field of art of disparity estimation dataset. Further, D’Angelo teaches the scene parameters comprise at least one of a lighting level (D’Angelo, page 10089, subsection B, second paragraph: “Illumination conditions and camera properties are studied in the 3-D environment to set the appropriate values for each city. The light is set to the Sun mode to have a homogeneous brightness in the whole area”), a resolution (D’Angelo, page 10090, subsection E. Description, last sentence: “All images have a resolution of 1024 × 1024 pixels”), a material, a texture, or a surface type. (***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to simply substitute the dataset of Duggal and Reyes with the dataset of D’Angelo; thus, one of ordinary skilled in the art would be motivated to make the substitution since Duggal and Reyes, and D’Angelo both use the dataset SyntCities (D’Angelo, Abstract: “we present SyntCities, a synthetic dataset resembling the aerial imagery on urban areas”. The Examiner notes D’Angelo and Reyes share the same authors).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 10
In regards to Claim 10, the combination of Duggal and Reyes teaches the apparatus of Claim 8.
The combination of Duggal and Reyes does not explicitly disclose the scene parameters comprise at least one of a lighting level, a resolution, a material, a texture, or a surface type.
D’Angelo is in the same field of art of disparity estimation dataset. Further, D’Angelo teaches the scene parameters comprise at least one of a lighting level (D’Angelo, page 10089, subsection B, second paragraph: “Illumination conditions and camera properties are studied in the 3-D environment to set the appropriate values for each city. The light is set to the Sun mode to have a homogeneous brightness in the whole area”), a resolution (D’Angelo, page 10090, subsection E. Description, last sentence: “All images have a resolution of 1024 × 1024 pixels”), a material, a texture, or a surface type. (***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to simply substitute the dataset of Duggal and Reyes with the dataset of D’Angelo; thus, one of ordinary skilled in the art would be motivated to make the substitution since Duggal and Reyes, and D’Angelo both use the dataset SyntCities (D’Angelo, Abstract: “we present SyntCities, a synthetic dataset resembling the aerial imagery on urban areas”. The Examiner notes D’Angelo and Reyes share the same authors).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 17
In regards to Claim 17, the combination of Duggal and Reyes teaches the medium of Claim 15.
The combination of Duggal and Reyes does not explicitly disclose the scene parameters comprise at least one of a lighting level, a resolution, a material, a texture, or a surface type.
D’Angelo is in the same field of art of disparity estimation dataset. Further, D’Angelo teaches the scene parameters comprise at least one of a lighting level (D’Angelo, page 10089, subsection B, second paragraph: “Illumination conditions and camera properties are studied in the 3-D environment to set the appropriate values for each city. The light is set to the Sun mode to have a homogeneous brightness in the whole area”), a resolution (D’Angelo, page 10090, subsection E. Description, last sentence: “All images have a resolution of 1024 × 1024 pixels”), a material, a texture, or a surface type. (***The Examiner notes since a listing with “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to simply substitute the dataset of Duggal and Reyes with the dataset of D’Angelo; thus, one of ordinary skilled in the art would be motivated to make the substitution since Duggal and Reyes, and D’Angelo both use the dataset SyntCities (D’Angelo, Abstract: “we present SyntCities, a synthetic dataset resembling the aerial imagery on urban areas”. The Examiner notes D’Angelo and Reyes share the same authors).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
CLAIM 21
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Duggal in view of Reyes, and further in view of Ohara et al. (US-20180285660-A1, hereinafter Ohara).
In regards to Claim 21, the combination of Duggal and Reyes teaches the method of Claim 1.
The combination does not explicitly disclose the learned stereo architecture is configured to shift features in a pair of images relative to one another to determine how well they match.
Ohara is in the same field of art of depth estimation using stereo images. Further, Ohara teaches the learned stereo architecture is configured to shift features in a pair of images relative to one another to determine how well they match. (Ohara, ¶ [0063-0067]: “the parallax calculator 23 designates two captured images respectively as the benchmark image and the reference image and partitions the reference image vertically and horizontally into blocks (small regions). The parallax calculator 23 performs matching while shifting a block of the partitioned reference image consecutively at equal pitch (for example, one pixel at a time) in the baseline length direction (the direction connecting the optical centers of the two cameras in the stereo camera) relative to the benchmark image … the parallax calculator 23 can use equation (3) to calculate the distance, Z, to an object of detection”, see equation (3). Ohara teaches using a matching algorithm to compute the distance to an imaged objects; the algorithm involves capturing 2 images, divide one image into small blocks, shift the divided block in one image to find matching block in the other image, determine the shift amount and use it to compute the distance)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Duggal and Reyes by incorporating parallax calculator system that is taught by Ohara, to make a depth estimation system that has a separate unit dedicated for the matching algorithm ; thus, one of ordinary skilled in the art would be motivated to combine the references since among its several aspects, the present invention recognizes there is a need to improve processing speed (Ohara, ¶ [0067]: “To perform one-dimensional matching between the benchmark image and the reference image at high speed, the parallax calculator 23 may include a dedicated parallel processing circuit for stereo image processing as a parallax calculation circuit”).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allowable Subject Matter
Claim 22 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NHUT HUY (JEREMY) PHAM whose telephone number is (703)756-5797. The examiner can normally be reached Mo - Fr. 8:30am - 6pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O'Neal Mistry can be reached on (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NHUT HUY (JEREMY) PHAMExaminerArt Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights