DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on March 25, 2026 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered by the examiner.
Amendments
This Office Action is in response to the amendment filed on March 25, 2026.
Claims 1, 9, and 10 have been amended.
Claim 5 has been cancelled.
No new claims have been added.
The objections and rejections from the prior correspondence that are not restated herein are withdrawn.
Response to Arguments
Applicant's arguments filed on March 25, 2026 have been fully considered.
Applicant's arguments regarding the 35 U.S.C. 101 rejections of the previous office action have been fully considered but are not persuasive. Applicant argues:
“Based on amended claim 1, Applicant respectfully argues that the process of configuring the training data and verification of the vertical disparity is an automated process and cannot be practically performed by human mind or mental process.”
Examiner respectfully disagrees. The claim, as drafted, does not recite limitations requiring the configuring of training data and verification of the vertical disparity to be performed automatically. Additionally, the claim does not require a specific time window for initiating and/or completing these processes. Therefore, under BRI, these limitations can be practically performed by the human mind as a mental process.
Applicant's arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered but are moot because the newly applied prior art reference of TRAN and YI, in combination with GAUSEBECK and ROWELL teaches the limitations of amended claims 1 and 9. The dependent claims 2-4, 6-8, and 10 are taught by various combinations of the references of GAUSEBECK, ROWELL, TRAN, YI, DONG, JIN, and/or PADFIELD, as shown in the 103 rejections below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4 and 6-10 are rejected under 35 U.S.C.101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-4 and 6-8 are directed to a process. Claims 9-10 are directed to a machine or an article of manufacture.
With respect to claim(s) 1 and 9:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
configuring/configure training data, based on the image data of the database, […] wherein the configuring of the training data comprises (Mental process – Configuring training data is defined as either clustering or rectification of the training data (see paragraph [0057]). Under broadest reasonable interpretation, rectification of data can be interpreted as the type of activity a human does through observation/judgment/opinion, such as correcting incorrect information, and clustering can be interpreted as the mental step of data organization into groups by mentally evaluating the similarities and/or differences of the data – see MPEP § 2106.04(a)(2)(III))
acquiring a rectification matrix; (Mathematical concepts – Acquiring a stereo rectification matrix involves mathematical calculations (see paragraph [0085]) – see MPEP § 2106.04(a)(2)(I))
applying the rectification matrix to the image data of the database; (Mathematical concepts – Applying a rectification matrix involves mathematical calculations (see paragraphs [0085-0086]) – see MPEP § 2106.04(a)(2)(I))
verifying a vertical disparity, based on a result of applying the rectification matrix, wherein the verifying the vertical disparity comprises: (Mental process – Verifying a vertical disparity of an image, such as the image shown in Fig. 7b, can be practically performed in the human mind – see MPEP § 2106.04(a)(2)(III))
calculating a numerical value of the vertical disparity of the result of applying the rectification matrix; (Mental process – A person can mentally calculate a numerical value of the vertical disparity. The claim defines the numerical value as “a value of a case in which a difference between feature points where disparity occurs is the greatest”, which means the maximum disparity value resulting from applying the matrix to image data – see MPEP § 2106.04(a)(2)(III))
determining whether the numerical value of the vertical disparity is less than a threshold value; (Mental process – A person can mentally determine a value is less than a threshold – see MPEP § 2106.04(a)(2)(III))
acquiring a second rectification matrix, based on the numerical value of the vertical disparity is not less than the threshold value, (Mathematical concepts – Acquiring a stereo rectification matrix involves mathematical calculations (see paragraph [0085]) – see MPEP § 2106.04(a)(2)(I))
wherein the numerical value of the vertical disparity is a value of a case in which a difference between feature points where disparity occurs is the greatest. (Mental process – Further description of the mental process of calculating a numerical value of the vertical disparity of the result of applying the rectification matrix – see MPEP § 2106.04(a)(2)(III))
If claim limitations, under their broadest reasonable interpretation, cover performance of the limitations as a mental process, but for the recitation of generic computer components, then the claim limitations fall within the mathematical or mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
(Claim 1) A data processing method for training a depth information estimation model, comprising: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 9) A data processing apparatus for training a depth information estimation model, comprising: a memory storing at least one program; and a processor configured to execute the at least one program to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
logging/log image data collected from one or more cameras; (Mere data gathering – Adding insignificant extra-solution activity of mere data gathering to the judicial exception – see § MPEP2106.05(g).)
transmitting/transmit the logged image data to a database; (Adding insignificant extra-solution activity to the judicial exception – see § MPEP2106.05(g).)
training/train a model, based on the training data, (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
(Claim 1) A data processing method for training a depth information estimation model, comprising: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
(Claim 9) A data processing apparatus for training a depth information estimation model, comprising: a memory storing at least one program; and a processor configured to execute the at least one program to: (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
logging/log image data collected from one or more cameras; (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(i) - Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).)
transmitting/transmit the logged image data to a database; (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(i) - Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).)
training/train a model, based on the training data. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim(s) 2:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein each piece of the image data collected from the one or more cameras comprises a plurality of images, wherein each of the plurality of images comprises a timestamp which indicates time information corresponding image is collected, and (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
the logging of the image data comprises pairing and storing images having a same timestamp. (Adding insignificant extra-solution activity to the judicial exception – see § MPEP2106.05(g).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein each piece of the image data collected from the one or more cameras comprises a plurality of images, wherein each of the plurality of images comprises a timestamp which indicates time information corresponding image is collected, and (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
the logging of the image data comprises pairing and storing images having a same timestamp. (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(iv) - Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 3:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
wherein the logging of the image data comprises storing each piece of the image data collected from the one or more cameras, in combination with inertia information measured by an inertial measurement unit. (Adding insignificant extra-solution activity to the judicial exception – see § MPEP2106.05(g).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the logging of the image data comprises storing each piece of the image data collected from the one or more cameras, in combination with inertia information measured by an inertial measurement unit. (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(iv) - Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 4:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
transmitting, to a display device, the image data collected from the one or more cameras in image units having the same timestamp, to display the image data collected from one or more cameras to a user. (Adding insignificant extra-solution activity to the judicial exception – see § MPEP2106.05(g).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
transmitting, to a display device, the image data collected from the one or more cameras in image units having the same timestamp, to display the image data collected from one or more cameras to a user. (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC) - see § MPEP 2106.05(d)(II)) - Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 6:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
extracting a feature vector for the image data of the database; (Mathematical concepts – Extracting a feature vector involves mathematical calculations (see paragraph [00106]) – see MPEP § 2106.04(a)(2)(I))
clustering the feature vector as one or more clusters; and (Mathematical concepts – Clustering involves mathematical calculations (see paragraph [00107]) – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
sampling the training data from the one or more clusters. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
sampling the training data from the one or more clusters. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 7:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
configuring evaluating data, based on the image data of the database; and (Mental process – Configuring evaluating data is done by either clustering or rectification of the evaluating data (see paragraph [0057]). Under broadest reasonable interpretation, rectification of data can be interpreted as the type of activity a human does through observation/judgment/opinion, such as correcting incorrect information, and clustering can be interpreted as the mental step of data organization into groups by mentally evaluating the similarities and/or differences of the data – see MPEP § 2106.04(a)(2)(III))
evaluating the model, based on the evaluating data. (Mathematical concepts – Evaluating the model involves mathematical calculations, such as calculating an evaluation index indicating the result of the evaluation (see paragraph [00126]). Additionally, per paragraph [0054], the index (e.g., evaluation index) may be a loss function for determining an optimum model parameter. – see MPEP § 2106.04(a)(2)(I))
Additionally, the claim(s) do not recite any new additional elements that would amount to an integration of the abstract idea into a practical application (individually or in combination) or significantly more than the judicial exception.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 8:
2A Prong 1: The claim(s) recite(s) an abstract idea. Specifically:
calculating an evaluation index, and (Mathematical concepts –Calculating an evaluation index involves mathematical calculations (see paragraph [00126]). Additionally, per paragraph [0054], the index (e.g., evaluation index) may be a loss function for determining an optimum model parameter. – see MPEP § 2106.04(a)(2)(I))
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
the data processing method further comprises storing the evaluation index in a model registry. (Adding insignificant extra-solution activity to the judicial exception – see § MPEP2106.05(g).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
the data processing method further comprises storing the evaluation index in a model registry. (Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception (WURC)- see MPEP § 2106.05(d)(ll)(iv) - Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
With respect to claim(s) 10:
2A Prong 2: The additional elements recited in the claim(s) do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
A non-transitory computer-readable recording medium having recorded thereon a program for executing the data processing method of claim 1 on a computer. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
2B: The claim(s) do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
A non-transitory computer-readable recording medium having recorded thereon a program for executing the data processing method of claim 1 on a computer. (Mere instructions to apply an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3 and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over GAUSEBECK (US 20190026958 A1) in view of ROWELL (US 20190208181 A1), TRAN ("Accurate Uncalibrated Rectification Using Feature Matching Re-Selection"), and YI ("An Epipolar Resampling Method for Multi-View High Resolution Satellite Images Based on Block"), hereafter GAUSEBECK, ROWELL, TRAN, and YI respectively.
Regarding Claim 1:
GAUSEBECK teaches:
A data processing method for training a depth information estimation model, comprising: (GAUSEBECK [0253] teaches: “However, relative to other 3D-from-2D models trained on conventional input data, the one or more optimized models 3328 can be configured to generate more precise and accurate depth derivation results based on training using the training data provided by the training data development component 3316.”)
logging image data collected from one or more cameras; (GAUSEBECK [0063] teaches: "[...] In some embodiments, the memory 122 can also store data received and/or generated by the computing device, such as (but not limited to), the received 2D image data 102 [...]" GAUSEBECK [0064] teaches: "For example, in some implementations, the reception component 108 can receive 2D image data 102 from one or more image capture devices (e.g., one or more cameras)."
transmitting the logged image data to a database; (GAUSEBECK [0132] teaches: "The user device 1402 can further store the 2D images and their associated derived depth data (e.g., in memory of the user device 1402, not shown), […]." GAUSEBECK [0276] teaches: "In this disclosure, terms such as “store,” “storage,” “data store,” data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component are utilized to refer to “memory components,” [...].")
configuring training data, based on the image data of the database; (GAUSEBECK [0245] teaches: "In one or more embodiments, the 3D-from-2D model development module 3314 can be configured to facilitate generating and/or training one or more 3D-from-2D models included in the 3D-from-2D model database 3326 based at least in part on data provided by the 3D space model database 3302. For example, in the embodiment shown, the 3D-from-2D model development module 3314 can include training data development component 3316 to facilitate gathering and/or generating training data (i.e., configuring training data) based on various types of rich 3D model information (discussed below) provided by the 3D space model database 3302 (i.e., based on the image data of the database). The 3D-from-2D model development module 3314 can further include model training component 3318, which can be configured to employ the training data to train and/or develop one or more 3D-from-2D neural network models included in the 3D-from-2D model database 3326." GAUSEBECK [0246] teaches: "In one or more embodiments, the 3D space model database 3302 can include a plethora of proprietary data associated with previously generated 3D space models that were generated using proprietary alignment techniques (e.g., those described herein), captured 2D image data, and associated captured depth data captured by various 3D sensors." GAUSEBECK [0249] teaches: "In one or more embodiments, the training data development module 3314 can extract this training data (e.g., indexed 2D images and associated 3D sensor data) from the 3D space model database 3302 for provision to the model training component 3318 to use in association with generating and/or training the one or more 3D-from-2D neural network models included in the 3D-from-2D model database 3326.")
training a model, based on the training data, (GAUSEBECK [0249] teaches: "Conventional training data used to generate 3D-from-2D neural network models (i.e. training a model) includes 2D images with known depth data (i.e., based on the training data) […]”)
GAUSEBECK is not relied upon for teaching:
wherein the configuring of the training data comprises:
acquiring a rectification matrix;
applying the rectification matrix to the image data of the database; and
verifying a vertical disparity, based on a result of applying the rectification matrix, wherein the verifying the vertical disparity comprises:
calculating a numerical value of the vertical disparity of the result of applying the rectification matrix;
determining whether the numerical value of the vertical disparity is less than a threshold value; and
acquiring a second rectification matrix, based on the numerical value of the vertical disparity is not less than the threshold value, and
wherein the numerical value of the vertical disparity is a value of a case in which a difference between feature points where disparity occurs is the greatest.
However, ROWELL teaches: wherein the configuring of the training data comprises: acquiring a rectification matrix; (ROWELL [0088] teaches: "The image rectification module 222, may generate rectification matrices for rectifying and projecting stereo images and or video frames using a variety of techniques. In one example, the image rectification module 222, calculates rectification transform matrices (R1, R2) and image projection matrices (P1, P2) for the left and right camera modules using the calibration metadata. The image rectification module 222 may calculate a right image projection matrix (P1) and a right rectification transform matrix (R1) from a camera rotation matrix (R), camera translation vector (T), and a right camera matrix (K1) containing the intrinsic calibration parameters for a right stereo camera module. Similarly, the image rectification module 222 may generate a left image projection matrix (P2) and left rectification transform matrix (R2) from a camera rotation matrix (R), a camera translation vector (T), and a left camera matrix (K2) containing intrinsic calibration parameters for a left stereo camera module. Rotation, projection transform, and projection matrices generated by the image rectification module 222 may be written in memory 213 or a storage device 211.")
applying a rectification matrix to the image data of the database; (ROWELL [0087] teaches: “Rectification is important for most applications of stereoscopic images or video sequences. For example, rectification is an essential preliminary step in producing depth information from stereo image disparity and projecting left and right stereo views as 3D. In some embodiments, four rectification matrices describe stereo camera rectification. Right and left rectification transform matrices (R1, R2) map the image planes of the left and right cameras on the same image plane, thereby making all the epipolar lines parallel and significantly simplifying stereo correspondence calculations. Right and left image projection matrices (P1, P2) describe the new rectified coordinate systems for the left and right camera modules.” ROWELL [0230] teaches: “In some embodiments, the image rectification module 222 transmits rectification matrices for projection to the remote streaming service 224, wherein a remote streaming client embeds the rectification matrices for projection in video and image files (i.e., image data of the database).” Examiner’s note: ROWELL’s rectification matrices can be applied to GAUSEBECK’s captured images.)
verifying a vertical disparity, based on a result of applying the rectification matrix, (ROWELL [0087] teaches: "[…] Applying the image projection matrices (P1, P2) to the coordinates of image points included in captured, unrectified right and left images projects the image points in rectified form as projection points. The image projection matrices P1, P2 are used to ensure (i.e., verifying) that the left and right images are vertically aligned (i.e., a vertical disparity), and satisfy an epipolar geometry. Once the stereo images or video sequences satisfy an epipolar geometry, depth information can be determined via disparity analysis and a 3D effect is observable in projected stereo content.")
[…] feature points […] (ROWELL [0082] teaches: " In one example, the total re-projection error is measured by determining the Euclidean distance between a projected image point and a measured image point for every point included in the calibration images." ROWELL [0087] teaches: "[…] Applying the image projection matrices (P1, P2) to the coordinates of image points (i.e., feature points) included in captured, unrectified right and left images projects the image points in rectified form as projection points.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK and ROWELL before them, to include ROWELL's rectification matrix and disparity analysis in GAUSEBECK's depth prediction method. One would have been motivated to make such a combination in order to generate more accurate depth information and resolve calibration errors over time and gradually improving the accuracy of the image rectification to avoid abrupt shifts in camera position and significant changes in camera performance (ROWELL [0024] and [0172]).
GAUSEBECK in view of ROWELL is not relied upon for teaching:
wherein the verifying the vertical disparity comprises:
calculating a numerical value of the vertical disparity of the result of applying the rectification matrix;
determining whether the numerical value of the vertical disparity is less than a threshold value; and
acquiring a second rectification matrix, based on the numerical value of the vertical disparity is not less than the threshold value, and
wherein the numerical value of the vertical disparity is a value of a case in which a difference between […] points where disparity occurs is the greatest.
However, TRAN teaches: wherein the verifying the vertical disparity comprises: calculating a numerical value of the vertical disparity of the result of applying the rectification matrix; (TRAN [page 142160, section A. Feature Matching Re-selection Module and Novel Pipeline] teaches: "The standard measurement to remove the outliers from feature matching process is vertical disparity error. The vertical disparity error of
m
l
i
,
m
r
i
matching pair is calculated as:
E
v
i
=
H
l
m
l
i
2
-
H
r
m
r
i
(
19
)
where
H
l
and
H
r
are outputs from the rectification module (i.e., of the result of applying the rectification matrix).")
determining whether the numerical value of the vertical disparity is less than a threshold value; (TRAN [page 142160, section A. Feature Matching Re-selection Module and Novel Pipeline] teaches: "Those pairs for which
E
v
i
>
T
v
e
r
_
d
i
s
p
_
e
r
r
were removed from the pool. In this work, we set
T
v
e
r
_
d
i
s
p
_
e
r
r
=
0.5
p
i
x
e
l
(i.e., threshold value).” Examiner’s note: TRAN teaches removing values that exceed
T
v
e
r
_
d
i
s
p
_
e
r
r
,and thus under BRI, determining whether the numerical value of the vertical disparity is less than a threshold value can be interpreted as determining the values that do not exceed the
T
v
e
r
_
d
i
s
p
_
e
r
r
value.)
acquiring a second rectification matrix, based on the numerical value of the vertical disparity is not less than the threshold value, (TRAN [page 142160, section A. Feature Matching Re-selection Module and Novel Pipeline] teaches: "Those pairs for which
E
v
i
>
T
v
e
r
_
d
i
s
p
_
e
r
r
were removed from the pool. In this work, we set
T
v
e
r
_
d
i
s
p
_
e
r
r
=
0.5
p
i
x
e
l
. The remaining pairs were fed back to the rectification module.
H
l
and
H
r
(i.e., second rectification matrix) from the rectification module were accepted if those conditions were satisfied:
E
v
J
=
0
,
o
r
M
1
<
10
(
20
)
where
E
v
J
=
1
M
1
∑
i
=
1
M
1
E
v
i
(
21
)
While the first condition stops FMR module when there is no outlier, the second condition works as a safe trigger to keep enough input correspondences for the rectification stage." Examiner’s note: TRAN teaches keeping pairs less than 0.5 pixel, and feeding back these pairs to acquire new rectification matrices
H
l
and
H
r
using the rectification module.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, and TRAN before them, to include TRAN’s outlier removal based on a threshold and feeding back iterations for the rectification stage in GAUSEBECK and ROWELL’s depth prediction method. One would have been motivated to make such a combination in order to remove outliers to prevent vertical disparity errors and unwanted geometric distortion and to secure satisfied vertical disparity errors (TRAN [Abstract]).
GAUSEBECK in view of ROWELL and TRAN is not relied upon for teaching, but YI teaches: wherein the numerical value of the vertical disparity is a value of a case in which a difference between […] points where disparity occurs is the greatest. (YI [page 162887, section A. Choose the block size and Divide Images Into blocks] teaches: "If we treat the epipolar curves as straight lines approximately, the vertical parallax of the conjugate points in the epipolar images will equal to the difference between the residuals in Figure 3 (c) and Figure 3 (d). We plot the difference in Figure 3 (e). Therefore, by estimating the maximum value of the difference between the fitting residuals, the maximum vertical parallax (i.e., the numerical value of the vertical disparity is a value of a case in which a difference between […] points where disparity occurs is the greatest) of the epipolar images can be estimated.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, TRAN, and YI before them, to include YI’s maximum vertical parallax estimation in GAUSEBECK, ROWELL, and TRAN’s depth prediction method One would have been motivated to make such a combination in order to limit the maximum vertical parallax in epipolar resampling (YI [Abstract]).
Regarding Claim 3:
GAUSEBECK in view of ROWELL, TRAN, and YI teaches the elements of claim 1 as outlined above. GAUSEBECK further teaches:
The data processing method of claim 1, wherein the logging of the image data comprises storing each piece of the image data collected from the one or more cameras, in combination with inertia information measured by an inertial measurement unit. (GAUSEBECK [0132] teaches: "In some implementations, the IMU measurements can be correlated via a timestamp or the like to respective images captured by the camera in association with movement of the camera during the capture process.”)
Regarding Claim 9:
The claim recites similar limitations as corresponding claim 1 and is rejected for similar reasons as claim 1 using similar teachings and rationale. GAUSEBECK further teaches:
a memory storing at least one program; and a processor configured to execute the at least one program to: (GAUSEBECK [0063] teaches: "The computing device 104 can include or be operatively coupled to at least one memory 104 and at least one processor 124. The at least one memory 122 can further store computer-executable instructions (e.g., the 3D model generation component 118, the 2D-from-3D processing module 106, one or more components of the 2D-from-3D processing module 106, and the navigation component 126) that when executed by the at least one processor 124 facilitate performance of operations defined by the computer-executable instructions. In some embodiments, the memory 122 can also store data received and/or generated by the computing device, such as (but not limited to), the received 2D image data 102, the derived 3D data 116, and the 3D model and alignment data 128. In other embodiments, the various data sources and data structures of system 100 (and other systems described herein) can be stored in other memory (e.g., at a remote device or system), that is accessible to the computing device 104 (e.g., via one or more networks).")
Regarding Claim 10:
GAUSEBECK in view of ROWELL, TRAN, and YI teaches the elements of claim 1 as outlined above. GAUSEBECK further teaches:
A non-transitory computer-readable recording medium having recorded thereon a program for executing the data processing method of claim 1 on a computer. (GAUSEBECK [0063] teaches: "The computing device 104 can include or be operatively coupled to at least one memory 104 and at least one processor 124. The at least one memory 122 can further store computer-executable instructions (e.g., the 3D model generation component 118, the 2D-from-3D processing module 106, one or more components of the 2D-from-3D processing module 106, and the navigation component 126) that when executed by the at least one processor 124 facilitate performance of operations defined by the computer-executable instructions. In some embodiments, the memory 122 can also store data received and/or generated by the computing device, such as (but not limited to), the received 2D image data 102, the derived 3D data 116, and the 3D model and alignment data 128. In other embodiments, the various data sources and data structures of system 100 (and other systems described herein) can be stored in other memory (e.g., at a remote device or system), that is accessible to the computing device 104 (e.g., via one or more networks).")
Claims 2 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over GAUSEBECK in view of ROWELL, TRAN, and YI as applied to claim 1 above, and further in view of DONG (CN 112074875 A), hereafter DONG.
Regarding Claim 2:
GAUSEBECK in view of ROWELL, TRAN, and YI teaches the elements of claim 1 as outlined above. GAUSEBECK further teaches:
The data processing method of claim 1, wherein each piece of the image data collected from the one or more cameras comprises a plurality of images, wherein each of the plurality of images comprises a timestamp which indicates time information corresponding image is collected, and the logging of the image data comprises […] storing images having a […] timestamp. (GAUSEBECK [0132] teaches: "In some implementations, the IMU measurements can be correlated via a timestamp or the like to respective images captured by the camera in association with movement of the camera during the capture process. For example, in an implementation in which the camera used to capture many images (i.e., each piece of the image data collected […] comprises a plurality of images) of an environment as the camera operator moves the camera to throughout the environment to different positions to capture different areas and perspective so the environment, each image (i.e., wherein each of the plurality of images) that is captured can be associated with a timestamp (i.e., comprises a timestamp) that indicates its relative time of capture (i.e., which indicates time information corresponding image is collected) to the other images, as well as motion data reflective of movement of the camera during and/or between captures." GAUSEBECK [0064] teaches: "For example, in some implementations, the reception component 108 can receive 2D image data 102 from one or more image capture devices (e.g., one or more cameras).” GAUSEBECK [0063] teaches: "[...] In some embodiments, the memory 122 can also store data received and/or generated by the computing device, such as (but not limited to), the received 2D image data 102 (i.e., […] storing images having a […] timestamp) [...].”)
However, GAUSEBECK in view of ROWELL, TRAN, and YI is not relied upon for teaching, but DONG teaches: pairing […] images having a same timestamp. (DONG [53] teaches: "[...] The time stamp of the image collected by each camera 112 is synchronized using a common clock, such as the real-time clock 134, so that the images collected by the camera 112 at the same time have the same time stamp." DONG [10] teaches: "In some examples, the method further includes selecting an image group from the images collected by the plurality of cameras, the image group including one captured image of each camera of the plurality of cameras, the image group being at the same time capture." DONG [56] teaches: "In operation 304, the processor 102 selects an image group from the captured images, and each of the plurality of cameras 112 used in the method 300 has a captured image, and these captured images are in the same Time captured. The images are selected based on the timestamp of each image so that the images in the group of images are captured at the same time.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, TRAN, YI, and DONG before them, to include DONG's timestamp encoding in GAUSEBECK, ROWELL, TRAN, and YI's depth prediction method. One would have been motivated to make such a combination so that the images collected by the camera at the same time are encoded to have the same time stamp that allows the images to be selected in the downstream process (DONG [53-54]).
Regarding Claim 4:
GAUSEBECK in view of ROWELL, TRAN, Yi, and DONG teaches the elements of claim 2 as outlined above. GAUSEBECK further teaches:
transmitting, to a display device, the image data collected from the one or more cameras in image units having the […] timestamp, to display the image data collected from one or more cameras to a user. (GAUSEBECK [0051] teaches: "The computer executable components can comprise a 3D data derivation component configured to employ one or more 3D-from-2D neural network models to derive 3D data from one or more 2D images captured of an object or environment from a current perspective of the object or environment viewed on or through a display of the device." GAUSEBECK [0087] teaches: "For example, in some implementations, a 3D model generated by the 3D model generation component 118, as well as the 2D images used to create the 3D model and the 3D information associated the 3D model can be stored in memory 122 (or another accessible memory device), and accessed by the user device (e.g., via a network using a browser, via a thin client application, etc.). In association with accessing the 3D model, the user device 130 can display (e.g. via display 132) an initial representation of the 3D model from a predefined initial perspective of a virtual camera relative to the 3D model.")
DONG further teaches: image units having the […] timestamp (DONG [53] teaches: "[...] The time stamp of the image collected by each camera 112 is synchronized using a common clock, such as the real-time clock 134, so that the images collected by the camera 112 at the same time have the same time stamp.")
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over GAUSEBECK in view of ROWELL, TRAN, and YI as applied to claim 1 above, and further in view of JIN (WO2021174513A1), hereafter JIN.
Regarding Claim 6:
GAUSEBECK in view of ROWELL, TRAN, and YI teaches the elements of claim 1 as outlined above. However, GAUSEBECK in view of ROWELL, TRAN, and YI is not relied upon for teaching, but JIN teaches:
wherein the configuring of the training data comprises: extracting a feature vector for the image data of the database; (JIN [107] teaches: “After the image data is collected, the data collection device 116 stores the training data in the database 113 […].” JIN [128] teaches: "For one frame of image, the image processing system/method of the present application It is possible to obtain features including image features, object features in the image, and object location information features at the same time, and obtain feature vectors based on these feature information, and obtain key frame images based on clustering and analysis of feature vectors.")
clustering the feature vector as one or more clusters; (JIN [16] teaches: "[…] the feature vectors are clustered to obtain a clustering result. It is possible to use, for example, K-means clustering (K-means) and centroid minimization cluster midpoint clustering. According to the clustering results, multiple cluster categories are obtained, each of the multiple cluster categories includes at least one image, the multiple cluster categories are sorted according to a set rule, and each of the multiple cluster categories is sorted.”)
sampling the training data from the one or more clusters. (JIN [16] teaches: “According to the clustering results, multiple cluster categories are obtained, each of the multiple cluster categories includes at least one image, the multiple cluster categories are sorted according to a set rule, and each of the multiple cluster categories is sorted. The first image after sorting is selected as the key frame, and the key frame is used as the training material of the object recognition algorithm.” JIN [78-82] teaches: "After the clustering is completed, the key frames can be selected (i.e., sampling) according to the following based on the clustering results.[…] select image 1 to represent cluster category 2, which means that image 1 as a whole can "represent" cluster category 2 this cluster image family. For cluster category 2 this cluster image family, the image 1 It can "represent" them as the key frame of this cluster of image families;" JIN [102] teaches: "The image frames/streams are processed in the cloud to obtain key frames. The key frames of can be used to train the neural network of the target detection algorithm.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, TRAN, YI, and JIN before them, to include JIN's feature vector extraction, clustering of feature vectors, and selecting images from the clusters that represents a cluster in GAUSEBECK, ROWELL, TRAN, and YI’s depth prediction method. One would have been motivated to make such a combination in order to fully consider the position information of the object in the image in the process of feature extraction, thus improving the accuracy of key frame acquisition (JIN [08]).
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over GAUSEBECK in view of ROWELL, TRAN, and YI as applied to claim 1 above, and further in view of JIN and PADFIELD (US 20250148365 A1), hereafter PADFIELD.
Regarding Claim 7:
GAUSEBECK in view of ROWELL, TRAN, and YI teaches the elements of claim 1 as outlined above. However, GAUSEBECK in view of ROWELL, TRAN, and YI is not relied upon for teaching, but JIN teaches:
configuring evaluating data, based on the image data of the database; (JIN [78-82] teaches: "After the clustering is completed, the key frames can be selected according to the following based on the clustering results.[…] select image 1 to represent cluster category 2, which means that image 1 as a whole can "represent" cluster category 2 this cluster image family. For cluster category 2 this cluster image family, the image 1 It can "represent" them as the key frame of this cluster of image families;" JIN [101] teaches: "Therefore, the use of the technical solution of this application can significantly eliminate redundant data, and these acquired key frames can be used for subsequent evaluations." Furthermore, paragraph [0100] discloses that configuring evaluating data may be performed by a clustering process of the data. Therefore, under broadest reasonable interpretation, configuring evaluating data can be interpreted as JIN’s feature vector clustering, which results in selected image key frames that can be used for subsequent evaluations.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, TRAN, YI, and JIN before them, to include JIN's clustering of feature vectors, and selecting images from the clusters that represents a cluster for evaluation in GAUSEBECK, ROWELL, TRAN, and YI’s depth prediction method. One would have been motivated to make such a combination in order to improve the accuracy of key frame acquisition and eliminate redundant data by using the key frames for subsequent evaluations (JIN [08] and [101]).
However, GAUSEBECK in view of ROWELL, TRAN, YI, and JIN is not relied upon for teaching, but PADFIELD teaches: evaluating the model, based on the evaluating data. (PADFIELD [0046] teaches: "The testing data can be fixed testing data or testing data (i.e., the evaluating data) sampled at the time of performance evaluation, for example using techniques described below with respect to 26." PADFIELD [0058] teaches: "In other implementations, the current set of testing data can be different set of testing data at each (or at least at some) iterations. For example, at each instance of operation 26, the method 12 can also include sampling, by the computing system, from the pool of data associated with the one or more ancillary systems to generate the current set of testing data." PADFIELD [0059] teaches: "Evaluating the performance of the updated model at 26 can include evaluating one or more performance measures. The performance measures can be any form of performance measures such as accuracy, precision, recall, regression metrics, etc. The performance measures can also include any number of statistical tests.")
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of GAUSEBECK, ROWELL, TRAN, YI, JIN, and PADFIELD before them, to include PADFIELD's calculating model evaluation results for later use in GAUSEBECK, ROWELL, TRAN, YI, and JIN's depth prediction method. One would have been motivated to make such a combination in order to evaluate the performance of the model over time on shifting data to select a model that truly provides the best performance (PADFIELD [0059]).
Regarding Claim 8:
GAUSEBECK in view of ROWELL, TRAN, YI, JIN, and PADFIELD teaches the elements of claim 7 as outlined above. PADFIELD further teaches:
wherein the evaluating of the model comprises calculating an evaluation index, (PADFIELD [0046] teaches: "At 18, the computing system can evaluate a performance of the current model (i.e., evaluating of the model) on the set of testing data and can store the performance results. [...] The performance measures can be any form of performance measures such as accuracy, precision, recall, regression metrics, etc (i.e., calculate an evaluation index). The performance measures can also include any number of statistical tests. The statistical tests can be evaluated on the output of the model itself or on various performance measures of the output of the model.")
and the data processing method further comprises storing the evaluation index in a model registry. (PADFIELD [0028] teaches: "[...] The performance evaluations can be stored for later use (e.g., at 28 as described below)." PADFIELD [0060] teaches: "At 28, the computing system can compare the performance of the updated model relative to the current set of training data with the respective performance of one or more other machine learning models on the current set of testing data or one or more past sets of testing data." Examiner's note: PADFIELD [0046] teaches evaluating the performance of a current model and storing the performance results for later use. PADFIELD [0060] teaches that the stored performance results can be used for comparing an updated model with the past model's performance. Under broadest reasonable interpretation, a person having ordinary skill in the art would recognize that storing an evaluation index in a model registry can be interpreted as the memory component in PADFIELD's computing system configured to store the model's performance results for comparison of model’s performance on changing data.)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Alvaro S Laham Bauzo whose telephone number is (571)272-5650. The examiner can normally be reached Mon-Fri 7:30 AM - 11:00 AM | 1:00 PM - 5:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S.L./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights