DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the pre-AIA first to invent provisions.
Response to Arguments
Applicant’s arguments, see remarks, filed 01/14/2026, with respect to 102 prior art rejection have been fully considered and are persuasive. The 102 rejection has been withdrawn. Please see below for new grounds of rejection based on a second Non-final.
Regarding applicant remarks based on 112a and 112b, have been fully considered and are persuasive. The 112a and 112b rejections has been withdrawn.
Regarding applicant remarks based on 101 abstract idea rejection, examiner disagrees. The claim recites multidimensional spaces, gradient descent, backpropagation, which the specification confirms as mathematical see at least paragraph 42-43, 50, 51, 55, 58-60 of the instant application.
Examiner suggests to what happens after the deployment of the model to integrate the mathematical concept into a practical application. For example ¶ 21 of the instant application states location prediction for beamforming or beam selection which would be connected to improving wireless communication rather than mathematical calculations. These amendments based on the language presented would overcome 101 and 103 rejections.
Claim Rejections - 35 USC § 101
11. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
12. Claims 1-30 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more.
Regarding Independent Claim 1:
Step 1: The claim is directed to a method, corresponding to a process, which is one of the statutory categories.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
backpropagating a gradient from one or more critic models to the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces;
to map scene data in the input data set to points in one or more multidimensional spaces;
Regarding the “backpropagating a gradient” recited in the claim, this backpropagation operation encompasses a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being
performed mentally or with a pen and paper.
Regarding the “to push the points in the one or more multidimensional spaces” recited in the claim, under the BRI, in light of the specification, this pushing points operation encompasses the mathematical concept of terms that influence the gradient formulation (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “map scene data… to points in one or more multidimensional spaces” recited in the claim, this mapping operation encompasses the mathematical concept of transformation of data points in multidimensional space (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small set of sample points, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional elements are adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
A computer-implemented method for training a machine learning model for predicting a location of an object in a multi-planar spatial environment,
training a generator model
The following additional elements add insignificant extra-solution activities (necessary data gathering and data storage and data transmission over a network) to the judicial exception [see MPEP 2106.05(d) and 2106.05(g)].
comprising: receiving an input data set of scene data;
and deploying at least the generator model
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
The following additional element is directed to storing and retrieving information in memory. The courts (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)) have recognized storing and retrieving information in memory as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) IV.].
comprising: receiving an input data set of scene data;
The following additional element is directed to receiving or transmitting data over a network. The courts (as per Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 / buySAFE, Inc. v. Google, Inc., 765 F.3d at 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)) have recognized receiving or transmitting data over a network as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) II.] and therefore fails to amount to significantly more than the judicial exception.
and deploying at least the generator model
Regarding Claim 2:
Step 1: The claim is directed to the process of claim 1.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises one or more images of a spatial environment
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible.] Therefore, claim 2 is not patent eligible.
Regarding Claim 3:
Step 1: The claim is directed to the process of claim 1.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 3 is not patent eligible.
Regarding Claim 4:
Step 1: The claim is directed to the process of claim 1.
Step 2A, Prong 1: The following limitations are directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the one or more critic models comprise a first critic model configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model
and a second critic model configured to enforce granularity among outputs of the generator model
Regarding the “promote co-planarity of points” and “enforce granularity” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0070] of the specification]). Given a sufficiently small set of gradient vector data, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 5:
Step 1: The claim is directed to the process of claim 4.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the first critic model is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane in the one or more multidimensional spaces for an input of scene data
Regarding the “select a best hypothesis of a plane” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of a single plane hypothesis function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0045-0046] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these hypothesis functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 6:
Step 1: The claim is directed to the process of claim 4.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the first critic model generates an attraction force and a repulsion force to apply to mapped data in the one or more multidimensional spaces
Regarding the “generates an attraction force and a repulsion force” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0078] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 7:
Step 1: The claim is directed to the process of claim 4.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the second critic model is configured to minimize a loss between adjacent clusters in a spatial environment
Regarding the “minimize a loss between adjacent clusters” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0074] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 8:
Step 1: The claim is directed to the process of claim 1.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
further comprising training the generator model and the one or more critic models
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 8 is not patent eligible.
Regarding Claim 9:
Step 1: The claim is directed to the process of claim 1.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the multidimensional spaces. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 9 is not patent eligible.
Regarding Independent Claim 10:
Step 1: The claim is directed to a method, corresponding to a process, which is one of the statutory categories.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
and mapping the scene data to a point in one or more multidimensional spaces through a generator model having a gradient backpropagated to the generator model from one or more critic models configured to separate points in the one or more multidimensional spaces into one of a plurality of planes
Regarding the “mapping the scene data to a point” recited in the claim, this mapping operation encompasses the mathematical concept of transformation of data points in multidimensional space (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small set of sample points, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “having a gradient backpropagated” recited in the claim, this backpropagation operation encompasses a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “separate the points in the one or more multidimensional spaces” recited in the claim, under the BRI, in light of the specification, this separating points operation encompasses the mathematical concept of terms that influence the gradient formulation (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
A computer-implemented method for predicting a location of an object in a multi-planar spatial environment,
The following additional element adds insignificant extra-solution activities (necessary data gathering and data storage) to the judicial exception [see MPEP 2106.05(g)].
comprising: receiving scene data;
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the calculated points in the multidimensional spaces. Therefore, the additional element does not integrate the abstract ideas into a practical application.
such that the points in the one or more multidimensional spaces are in a vicinity of any of the plurality of planes in the one or more multidimensional spaces
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
The following additional element is directed to storing and retrieving information in memory. The courts (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)) have recognized storing and retrieving information in memory as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) IV.].
comprising: receiving scene data;
Regarding Claim 11:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
further comprising predicting a location on a plane in the one or more multidimensional spaces
Regarding the “predicting a location on a plane” recited in the claim, under the BRI, in light of the specification, this prediction operation encompasses the mathematical concept of triangulation and distance measurements (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0022] of the specification]). Given a sufficiently small set of coordinates or points in a multidimensional space, nothing in the claim prohibits this prediction of a location from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data for prediction location. Therefore, the additional element does not integrate the abstract ideas into a practical application.
at which the received scene data is located based on the point in the one or more multidimensional spaces to which the scene data is mapped
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 11 is not patent eligible.
Regarding Claim 12:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 10.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises one or more images of a spatial environment
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 12 is not patent eligible.
Regarding Claim 13:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 10.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 13 is not patent eligible.
Regarding Claim 14:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The following limitations are directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane
and a repulsion force that pushes points located on different planes in the one or more multidimensional spaces away from each other
Regarding the “comprises an attraction force and a repulsion force term that pushes co-planar points” and “a repulsion force that pushes points” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0078] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception. Therefore, claim 14 is not patent eligible.
Regarding Claim 15:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment
Regarding the “minimized loss between adjacent cluster” recited in the claim, under the BRI, in light of the specification, this operation encompasses the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0074] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits this loss function from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception. Therefore, claim 15 is not patent eligible.
Regarding Claim 16:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 10.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 16 is not patent eligible.
Regarding Claim 17:
Step 1: The claim is directed to the process of claim 10.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 10.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the one or more multidimensional spaces comprise a three-dimensional space and a 128-dimensional space
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 17 is not patent eligible.
Regarding Independent Claim 18:
Step 1: The claim is directed to a processing system, corresponding to a system, which is one of the statutory categories.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
backpropagate a gradient from one or more critic models to the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces;
to map scene data in the input data set to points in one or more multidimensional spaces;
Regarding the “backpropagate a gradient” recited in the claim, this backpropagation operation encompasses a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being
performed mentally or with a pen and paper.
Regarding the “to push the points in the one or more multidimensional spaces” recited in the claim, under the BRI, in light of the specification, this pushing points operation encompasses the mathematical concept of terms that influence the gradient formulation (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “map scene data… to points in one or more multidimensional spaces” recited in the claim, this mapping operation encompasses the mathematical concept of transformation of data points in multidimensional space (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small set of sample points, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
train a generator model
The following additional elements add insignificant extra-solution activities (necessary data gathering and data storage and data transmission over a network) to the judicial exception [see MPEP 2106.05(d) and 2106.05(g)].
receive an input data set of scene data;
and deploy at least the generator model
Claim 1 recites the additional elements: “A processing system, comprising: a memory having executable instructions stored thereon; and a processor configured to execute the executable instructions to cause the processing system to:”, which are recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., a processing system), or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
The following additional element is directed to storing and retrieving information in memory. The courts (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)) have recognized storing and retrieving information in memory as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) IV.].
receive an input data set of scene data;
The following additional element is directed to receiving or transmitting data over a network. The courts (as per Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 / buySAFE, Inc. v. Google, Inc., 765 F.3d at 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014)) have recognized receiving or transmitting data over a network as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) II.] and therefore fails to amount to significantly more than the judicial exception.
and deploy at least the generator model
Regarding Claim 19:
Step 1: The claim is directed to the system of claim 18.
Step 2A, Prong 1: The following limitations are directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the one or more critic models comprise a first critic model1 configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model
and a second critic model2 configured to enforce granularity among outputs of the generator model
Regarding the “promote co-planarity of points” and “enforce granularity” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0070] of the specification]). Given a sufficiently small set of gradient vector data, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 20:
Step 1: The claim is directed to the system of claim 19.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the first critic model is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane in the one or more multidimensional spaces for an input of scene data
Regarding the “select a best hypothesis of a plane” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of a single plane hypothesis function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0045-0046] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these hypothesis functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 21:
Step 1: The claim is directed to the system of claim 19.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the second critic model3 is configured to minimize a loss between adjacent clusters in a spatial environment
Regarding the “minimize a loss between adjacent clusters” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0074] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
Regarding Claim 22:
Step 1: The claim is directed to the system of claim 18.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 18.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
wherein the processor is further configured to cause the processing system to train the generator model and the one or more critic models
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 22 is not patent eligible.
Regarding Independent Claim 23:
Step 1: The claim is directed to a processing system, corresponding to a system, which is one of the statutory categories.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
and map the scene data to a point in one or more multidimensional spaces through a generator model having a gradient backpropagated to the generator model from one or more critic models4 configured to separate points in the one or more multidimensional spaces into one of a plurality of planes
Regarding the “map the scene data to a point” recited in the claim, this mapping operation encompasses the mathematical concept of transformation of data points in multidimensional space (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small set of sample points, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “having a gradient backpropagated” recited in the claim, this backpropagation operation encompasses a mathematical concept (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Regarding the “separate the points in the one or more multidimensional spaces” recited in the claim, under the BRI, in light of the specification, this separating points operation encompasses the mathematical concept of terms that influence the gradient formulation (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations). Given a sufficiently small gradient vector of partial derivatives, nothing in the claim prohibits this process from being performed mentally or with a pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional elements are adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the judicial exception into a practical application.
A computer-implemented method for predicting a location of an object in a multi-planar spatial environment,
The following additional element adds insignificant extra-solution activities (necessary data gathering and data storage) to the judicial exception [see MPEP 2106.05(g)].
receive scene data;
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the calculated points in the multidimensional spaces. Therefore, the additional element does not integrate the abstract ideas into a practical application.
such that the points in the one or more multidimensional spaces are in a vicinity of any of the plurality of planes in the one or more multidimensional spaces
Claim 1 recites the additional elements: “A processing system, comprising: a memory having executable instructions stored thereon; and a processor configured to execute the executable instructions to cause the processing system to:”, which are recited at a high level of generality as mere instructions to implement an abstract idea on a computer (i.e., a processing system), or merely use a computer as a tool to perform an abstract idea (i.e., as generic computer components performing generic computer functions). See MPEP 2106.05(f).
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
The following additional element is directed to storing and retrieving information in memory. The courts (Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)) have recognized storing and retrieving information in memory as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity to the judicial exception [see MPEP 2106.05(d) IV.].
receive scene data;
Regarding Claim 24:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the processor is further configured to cause the processing system to predict a location on a plane in the one or more multidimensional spaces
Regarding the “predict a location on a plane” recited in the claim, under the BRI, in light of the specification, this prediction operation encompasses the mathematical concept of triangulation and distance measurements (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0022] of the specification]). Given a sufficiently small set of coordinates or points in a multidimensional space, nothing in the claim prohibits this prediction of a location from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data for prediction location. Therefore, the additional element does not integrate the abstract ideas into a practical application.
at which the received scene data is located based on the point in the one or more multidimensional spaces to which the scene data is mapped
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 24 is not patent eligible.
Regarding Claim 25:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 23.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises one or more images of a spatial environment
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 25 is not patent eligible.
Regarding Claim 26:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 23.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the scene data. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 26 is not patent eligible.
Regarding Claim 27:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The following limitations are directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane
and a repulsion force that pushes points located on different planes in the one or more multidimensional spaces away from each other
Regarding the “comprises an attraction force and a repulsion force term that pushes co-planar points” and “a repulsion force that pushes points” recited in the claim, under the BRI, in light of the specification, these operations encompass the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0078] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits these loss functions from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception. Therefore, claim 27 is not patent eligible.
Regarding Claim 28:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The following limitation is directed to the abstract idea of a mathematical concept [see MPEP 2106.04(a)(2) I. C.]. In particular, the claim recites mathematical processes that can be performed in the human mind or with pen and paper (including an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic operation such as exponentiation).
wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment
Regarding the “minimized loss between adjacent cluster” recited in the claim, under the BRI, in light of the specification, this operation encompasses the mathematical concept of mathematical terms in a loss function (i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations), (see, e.g., paragraph [0074] of the specification]). Given a sufficiently small set of sample points, nothing in the claim prohibits this loss function from being performed mentally or with pen and paper.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception. Therefore, claim 28 is not patent eligible.
Regarding Claim 29:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 23.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 29 is not patent eligible.
Regarding Claim 30:
Step 1: The claim is directed to the system of claim 23.
Step 2A, Prong 1: The claim recites the same abstract ideas as in claim 23.
Step 2A, Prong 2: There are no additional elements in this claim that integrate the judicial exception into a practical application.
The following additional element does not meaningfully limit the judicial exception [see MPEP 2106.05(e)]. The claim simply recites additional information regarding the characteristics of the loss function used to train the generator and critic models. Therefore, the additional element does not integrate the abstract ideas into a practical application.
wherein the one or more multidimensional spaces comprise a three-dimensional space and a 128-dimensional space
Step 2B: There are no additional elements in this claim that amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional limitation amounts to no more than using generic computer components to implement the exception. Limiting the abstract idea to a particular technological context or field of use, does not render the claim patent eligible. Therefore, claim 30 is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 10-12, 18, and 23-25 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela (US 20230004760 A1) in view of Shan et al. (“InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs”, 2020).
Regarding Claim 1,
Mustikovela discloses a computer-implemented method for training a machine learning model for predicting a location of an object in a multi-planar spatial environment, comprising (see, e.g., paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose as input and generates an image used to train object detection neural network. In at least one embodiment, a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose" [i.e., a GAN is trained to detect objects and their location and pose (prediction location)] and paragraph [0195]: “In at least one embodiment, real-time ray-tracing hardware accelerator may be used to quickly and efficiently determine positions and extents of objects (e.g., within a world model), to generate real-time visualization simulations, for RADAR signal interpretation… for comparison to LIDAR data for purposes of localization and/or other functions, and/or for other uses" [i.e., ray-tracing is used in a world model with RADAR and LIDAR acts as a multi-planar spatial environment]):
receiving an input data set of scene data (see, e.g., paragraph [0058]: “Generative Adversarial Networks (“GANs”) are used which provide control using input parameters like shape, viewpoint, position and key points opening up a possibility of synthesizing images with desired attributes" and paragraph [0077]: “input to a network 502, 506 consists of a style vector zf and an object pose in camera coordinates (vf, lf) for one or more foreground objects. In at least one embodiment, value of vf represents azimuth of an object and lf represents horizontal and depth translation. In at least one embodiment, a style vector zb and pose (vb, lf) is input to a second network 504 for background" [i.e., the input data to the network is scene data of foreground and background objects]);
training a generator model to map scene data in the input data set to points in one or more multidimensional spaces (see, e.g., paragraph [0072]: “Generative Adversarial Networks (“GANs”) are used which provide control using input parameters like shape, viewpoint, position and key points opening up a possibility of synthesizing images with desired attributes”, paragraph [0074]: “input to a network 502, 506 consists of a style vector zf and an object pose in camera coordinates (vf, lf) for one or more foreground objects. In at least one embodiment, value of vf represents azimuth of an object and lf represents horizontal and depth translation. In at least one embodiment, a style vector zb and pose (vb, lf) is input to a second network 504 for background” and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image” [i.e., the localization of an object with depth and pose parameters in a synthesized image is mapping scene data to points in multidimensional space]);
backpropagating a gradient from one or more critic models to the generator model (see, e.g., paragraph [0073]: “object detection adaptation module is designed to provide feedback to synthesis network S 202 to optimally adapt it to downstream task of object detection… it tightly couples object detector custom-character 302 and synthesizer S 202 for joint end-to-end training and also introduces specific losses 306 and 308 to guide synthesis process towards better object detection learning… a multi-scale discriminator network is used to train the system… a loss generated to train detection network is also used to train a generator network” [i.e., a loss used to provide feedback and adapt the generator functions, discloses gradient descent being backpropagated from the discriminator network to the generator network in the context of training these machine learning models], also see ¶ 120, “code and/or data storage 805 stores weight parameters and/or input/output data of each layer of a neural network trained or used in conjunction with one or more embodiments during backward propagation of input/output data and/or weight parameters during training and/or inferencing using aspects of one or more embodiments”, also see ¶ 130, “errors are then propagated back through untrained neural network 906. In at least one embodiment, training framework 904 adjusts weights that control untrained neural network 906. In at least one embodiment, training framework 904 includes tools to monitor how well untrained neural network 906 is converging towards a model, such as trained neural network 908, suitable to generating correct answers, such as in result 914, based on input data such as a new dataset 912.”, also see ) to push the points in the one or more multidimensional spaces (see, e.g., paragraph [0078]: “In at least one embodiment, resulting 3D features of objects are collated using an element-wise maximum operation and then projected 514 onto 2D using a perspective camera transformation followed by a set of 2D convolutions 516 to yield I.sub.g 518” [i.e., the projection of 3D features of objects onto 2D during a convolution functions as pushing the points of in the multidimensional space to a plurality of planes in 2D] and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (v.sub.f, l.sub.f). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it. In at least one embodiment, a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (v.sub.f, l.sub.f) via perspective projection”);
and deploying at least the generator model (see, e.g., paragraph [0051]: “FIG. 38 is a system diagram for an example system for training, adapting, instantiating and deploying machine learning models in an advanced computing pipeline, in accordance with at least one embodiment” and paragraph [0533]: “In at least one embodiment, process 3700 may be executed within a training system 3704 and/or a deployment system 3706. In at least one embodiment, training system 3704 may be used to perform training, deployment, and implementation of machine learning models (e.g., neural networks, object detection algorithms, computer vision algorithms, etc.) for use in deployment system 3706”).
Under the broadest reasonable interpretation (BRI), any model described in the specification of Mustikovela can reasonably be deployed via the deployment system described herein, including “at least the generator model” as claimed.
Mustikovela do not specifically teach the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces.
Shen teaches the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces (see page 3, section 3.1, “Given a well-trained GAN model, the generator can be viewed as a deterministic function g : Z → X. Here, Z ⊆Rd denotes the d-dimensional latent space, for which Gaussian distribution N(0,Id) is commonly used [1], [3], [18], [20]. X stands for the image space, where each sample x possesses certain semantic information, like gender and age for face model. Suppose we have a semantic scoring function fS : X → S, where S ⊆ Rm represents the semantic space with m semantics. We can bridge the latent space Z and the semantic space S with s = fS(g(z)), where s and z denote semantic scores and the sampled latent code respectively.”, also see page 4, section 4, “we apply InterFaceGAN to interpret the face representation learned by state-of-the-art GAN models, i.e., PGGAN [1] and StyleGAN [3], both of which can produce high-quality faces with 1024 × 1024 resolution. PGGAN is a representative of the traditional generator where the latent code is only fed into the very first convolutional layer. By contrast, StyleGAN proposed a style-based generator, which first maps the latent code from latent space Z to a disentangled latent space W before applying it for the generation. In addition, the disentangled latent code is fed to all convolutional layers.”, also see page 6, section 4.2.1, “We surprisingly find that GAN also encodes such quality information in the latent space. Based on this discovery, we manage to correct some mistakes GAN has made in the generation process by moving the latent code towards the positive “quality” direction, as shown in Figure 7.”, also see section 4.2.2).
Both Mustikovela and Shen pertain to the problem of Generative Adversarial Networks (GANs) training, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Mustikovela and Shen to teach the above limitations. The motivation for doing so would be “we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space. We first find that GANs learn various semantics in some linear subspaces of the latent space. After identifying these subspaces, we can realistically manipulate the corresponding facial attributes without retraining the model. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection, resulting in more precise control of the attribute manipulation. Besides manipulating the gender, age, expression, and presence of eyeglasses, we can even alter the face pose and fix the artifacts accidentally made by GANs. Furthermore, we perform an in-depth face identity analysis and a layer-wise analysis to evaluate the editing results quantitatively. Finally, we apply our approach to real face editing by employing GAN inversion approaches and explicitly training feed-forward models based on the synthetic data established by InterFaceGAN. Extensive experimental results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable face representation.” (see Shen Abstract).
Regarding claim 2, as discussed above Mustikovela and Shen teaches the method of claim 1. Mustikovela further discloses wherein the scene data comprises one or more images of a spatial environment (see, e.g., paragraph [0071]: “In at least one embodiment, a goal is to learn a detection network… which best detects objects (such as cars) in a target domain (such as out-door driving scenes from a city)” and paragraph [0336]: “to accelerate tiling operations for tile-based rendering, in which rendering operations for a scene are subdivided in image space, for example to exploit local spatial coherence within a scene or to optimize use of internal caches").
Regarding Independent Claim 10,
Mustikovela discloses a computer-implemented method for predicting a location of an object in a multi-planar spatial environment, comprising (see, e.g., Mustikovela paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose as input and generates an image used to train object detection neural network. In at least one embodiment, a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose” and paragraph [0195]: “In at least one embodiment, real-time ray-tracing hardware accelerator may be used to quickly and efficiently determine positions and extents of objects (e.g., within a world model), to generate real-time visualization simulations, for RADAR signal interpretation… for comparison to LIDAR data for purposes of localization and/or other functions, and/or for other uses” [i.e., ray-tracing is used in a world model with RADAR and LIDAR acts as a multi-planar spatial environment]):
receiving scene data; and mapping the scene data to a point in one or more multidimensional spaces through a generator model (see, e.g., Mustikovela paragraph [0072]: “In at least one embodiment, S is modeled by a pose-aware generator 202, which synthesizes images {I.sub.g} 204 of objects conditioned on pose parameters 206 (viewpoint (v), location (l), and appearance (z)) and obtain 2D bounding box annotations {A.sub.g} 208 for them”, paragraph [0077]: “input to a network 502, 506 consists of a style vector zf and an object pose in camera coordinates (vf, lf) for one or more foreground objects. In at least one embodiment, value of vf represents azimuth of an object and lf represents horizontal and depth translation. In at least one embodiment, a style vector zb and pose (vb, lf) is input to a second network 504 for background” and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image” [i.e., the localization of an object with depth and pose parameters in a synthesized image is analogous to mapping scene data to points in multidimensional space using synthesis network S (a generator model)])
having a gradient backpropagated to the generator model from one or more critic models (see, e.g., Mustikovela paragraph [0073]: “object detection adaptation module is designed to provide feedback to synthesis network S 202 to optimally adapt it to downstream task of object detection… it tightly couples object detector custom-character 302 and synthesizer S 202 for joint end-to-end training and also introduces specific losses 306 and 308 to guide synthesis process towards better object detection learning… a multi-scale discriminator network is used to train the system… a loss generated to train detection network is also used to train a generator network” [i.e., a loss used to provide feedback and adapt the generator functions, discloses gradient descent being backpropagated from the discriminator network to the generator network in the context of training these machine learning models])
configured to separate points in the one or more multidimensional (see, e.g., Mustikovela paragraph [0078]: “In at least one embodiment, resulting 3D features of objects are collated using an element-wise maximum operation and then projected 514 onto 2D using a perspective camera transformation followed by a set of 2D convolutions 516 to yield I.sub.g 518” [i.e., the projection of 3D features of objects onto 2D during a convolution functions as pushing the points of in the multidimensional space to a plurality of planes in 2D] and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (v.sub.f, l.sub.f). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it. In at least one embodiment, a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (v.sub.f, l.sub.f) via perspective projection” [i.e., the system computes bounding boxes by projecting 3D object coordinates onto a 2D image plane, constraining (separating) the points to 2D plane regions)])
such that the points in the one or more multidimensional spaces are in a vicinity of any of the plurality of planes in the one or more multidimensional spaces (see, e.g., Mustikovela paragraph [0081]: “synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it… a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (vf, lf) via perspective projection… At least one embodiment obtains a 2D bounding box Ag for synthesized image Ig by computing maximum and minimum coordinates of projected 3D bounding box in image plane” [i.e., the coordinates (points) of the projected 3D bounding box in the image plane indicates the points dispersed in the vicinity of image planes in the 3D bounding box latent space]).
Mustikovela do not specifically teach the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces.
Shen teaches the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces (see page 3, section 3.1, “Given a well-trained GAN model, the generator can be viewed as a deterministic function g : Z → X. Here, Z ⊆Rd denotes the d-dimensional latent space, for which Gaussian distribution N(0,Id) is commonly used [1], [3], [18], [20]. X stands for the image space, where each sample x possesses certain semantic information, like gender and age for face model. Suppose we have a semantic scoring function fS : X → S, where S ⊆ Rm represents the semantic space with m semantics. We can bridge the latent space Z and the semantic space S with s = fS(g(z)), where s and z denote semantic scores and the sampled latent code respectively.”, also see page 4, section 4, “we apply InterFaceGAN to interpret the face representation learned by state-of-the-art GAN models, i.e., PGGAN [1] and StyleGAN [3], both of which can produce high-quality faces with 1024 × 1024 resolution. PGGAN is a representative of the traditional generator where the latent code is only fed into the very first convolutional layer. By contrast, StyleGAN proposed a style-based generator, which first maps the latent code from latent space Z to a disentangled latent space W before applying it for the generation. In addition, the disentangled latent code is fed to all convolutional layers.”, also see page 6, section 4.2.1, “We surprisingly find that GAN also encodes such quality information in the latent space. Based on this discovery, we manage to correct some mistakes GAN has made in the generation process by moving the latent code towards the positive “quality” direction, as shown in Figure 7.”, also see section 4.2.2).
Both Mustikovela and Shen pertain to the problem of Generative Adversarial Networks (GANs) training, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Mustikovela and Shen to teach the above limitations. The motivation for doing so would be “we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space. We first find that GANs learn various semantics in some linear subspaces of the latent space. After identifying these subspaces, we can realistically manipulate the corresponding facial attributes without retraining the model. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection, resulting in more precise control of the attribute manipulation. Besides manipulating the gender, age, expression, and presence of eyeglasses, we can even alter the face pose and fix the artifacts accidentally made by GANs. Furthermore, we perform an in-depth face identity analysis and a layer-wise analysis to evaluate the editing results quantitatively. Finally, we apply our approach to real face editing by employing GAN inversion approaches and explicitly training feed-forward models based on the synthetic data established by InterFaceGAN. Extensive experimental results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable face representation.” (see Shen Abstract).
Regarding claim 11, as discussed above Mustikovela and Shen teaches the method of claim 10. Mustikovela further discloses further comprising predicting a location on a plane in the one or more multidimensional spaces (see, e.g., paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose as input and generates an image used to train object detection neural network. In at least one embodiment, a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose” [i.e., the object detection model trains an object detection model that calculates a loss and pose (spatial information in a geometric space) for the location of objects (predicting the location of an objects) in an image which is a prediction on a 2D plane])
at which the received scene data is located based on the point in the one or more multidimensional spaces to which the scene data is mapped (see, e.g., paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose… a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose ” [i.e., the predicted object location/pose at a point in latent space is directly derived from the objects location and pose input (scene data) in the 2D image plane]).
Regarding claim 12, above Mustikovela and Shen teaches the method of claim 10. Mustikovela further discloses wherein the scene data comprises one or more images of a spatial environment (see, e.g., paragraph(s) [0071]: “In at least one embodiment, a goal is to learn a detection network Figure ‘F’, which best detects objects (such as cars) in a target domain (such as out-door driving scenes from a city). At least one embodiment further assumes availability of an unlabeled image collection {It} from target domain each containing an unknown number of objects per image (see FIG. 1 ). To train Figure ‘F’, at least one embodiment leverages object images and their bounding box annotations synthesized by a controllable generative network S 202 " [i.e., the target domain out-door driving scenes from a city includes images from a spatial environment]).
Regarding Independent claim 18,
Mustikovela discloses a processing system, comprising: a memory having executable instructions stored thereon (see, e.g., paragraph [0111]: “In at least one embodiment, an object detection neural network includes a controllable GAN, an object detection network, and one or more discriminative networks that produce one or more losses… object detection neural network can be implemented with one or more computer systems having one or more processors such as those described below. In at least one embodiment, as a result of executing instructions stored on computer-readable memory, on one or more processors cause a computer system to perform operations described below”);
and a processor configured to execute the executable instructions to cause the processing system to: receive an input data set of scene data (see, e.g., paragraph [0111]: “In at least one embodiment, as a result of executing instructions stored on computer-readable memory, on one or more processors cause a computer system to perform operations described below” and paragraph [0541]: “In at least one embodiment, a data processing pipeline may receive input data (e.g., imaging data 3708)… In at least one embodiment, input data may be representative of one or more images, video, and/or other data representations generated by one or more imaging devices");
train a generator model to map scene data in the input data set to points in one or more multidimensional spaces (see, e.g., paragraph [0072]: “Generative Adversarial Networks (“GANs”) are used which provide control using input parameters like shape, viewpoint, position and key points opening up a possibility of synthesizing images with desired attributes” [i.e., ] and paragraph [0074]: “input to a network 502, 506 consists of a style vector zf and an object pose in camera coordinates (vf, lf) for one or more foreground objects. In at least one embodiment, value of vf represents azimuth of an object and lf represents horizontal and depth translation. In at least one embodiment, a style vector zb and pose (vb, lf) is input to a second network 504 for background” and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image” [i.e., the localization of an object with depth and pose parameters in a synthesized image is analogous to mapping scene data to points in multidimensional space]);
backpropagate a gradient from one or more critic models to the generator model (see, e.g., paragraph [0073]: “object detection adaptation module is designed to provide feedback to synthesis network S 202 to optimally adapt it to downstream task of object detection… it tightly couples object detector custom-character 302 and synthesizer S 202 for joint end-to-end training and also introduces specific losses 306 and 308 to guide synthesis process towards better object detection learning… a multi-scale discriminator network is used to train the system… a loss generated to train detection network is also used to train a generator network” [i.e., a loss used to provide feedback and adapt the generator functions, is gradient descent being backpropagated from the discriminator network to the generator network in the context of training these machine learning models])
to push the points in the one or more multidimensional (see, e.g., paragraph [0078]: “In at least one embodiment, resulting 3D features of objects are collated using an element-wise maximum operation and then projected 514 onto 2D using a perspective camera transformation followed by a set of 2D convolutions 516 to yield I.sub.g 518” [i.e., the projection of 3D features of objects onto 2D during a convolution functions as pushing the points of in the multidimensional space to a plurality of planes in 2D] and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (v.sub.f, l.sub.f). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it. In at least one embodiment, a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (v.sub.f, l.sub.f) via perspective projection”).
and deploy at least the generator model (see, e.g., paragraph [0051]: “FIG. 38 is a system diagram for an example system for training, adapting, instantiating and deploying machine learning models in an advanced computing pipeline, in accordance with at least one embodiment” and paragraph [0533]: “In at least one embodiment, process 3700 may be executed within a training system 3704 and/or a deployment system 3706. In at least one embodiment, training system 3704 may be used to perform training, deployment, and implementation of machine learning models (e.g., neural networks, object detection algorithms, computer vision algorithms, etc.) for use in deployment system 3706”).
Under the broadest reasonable interpretation (BRI), any model described in the specification of Mustikovela can reasonably be deployed via the deployment system described herein, including one of the generator models, as claimed.
Mustikovela do not specifically teach the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces.
Shen teaches the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces (see page 3, section 3.1, “Given a well-trained GAN model, the generator can be viewed as a deterministic function g : Z → X. Here, Z ⊆Rd denotes the d-dimensional latent space, for which Gaussian distribution N(0,Id) is commonly used [1], [3], [18], [20]. X stands for the image space, where each sample x possesses certain semantic information, like gender and age for face model. Suppose we have a semantic scoring function fS : X → S, where S ⊆ Rm represents the semantic space with m semantics. We can bridge the latent space Z and the semantic space S with s = fS(g(z)), where s and z denote semantic scores and the sampled latent code respectively.”, also see page 4, section 4, “we apply InterFaceGAN to interpret the face representation learned by state-of-the-art GAN models, i.e., PGGAN [1] and StyleGAN [3], both of which can produce high-quality faces with 1024 × 1024 resolution. PGGAN is a representative of the traditional generator where the latent code is only fed into the very first convolutional layer. By contrast, StyleGAN proposed a style-based generator, which first maps the latent code from latent space Z to a disentangled latent space W before applying it for the generation. In addition, the disentangled latent code is fed to all convolutional layers.”, also see page 6, section 4.2.1, “We surprisingly find that GAN also encodes such quality information in the latent space. Based on this discovery, we manage to correct some mistakes GAN has made in the generation process by moving the latent code towards the positive “quality” direction, as shown in Figure 7.”, also see section 4.2.2).
Both Mustikovela and Shen pertain to the problem of Generative Adversarial Networks (GANs) training, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Mustikovela and Shen to teach the above limitations. The motivation for doing so would be “we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space. We first find that GANs learn various semantics in some linear subspaces of the latent space. After identifying these subspaces, we can realistically manipulate the corresponding facial attributes without retraining the model. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection, resulting in more precise control of the attribute manipulation. Besides manipulating the gender, age, expression, and presence of eyeglasses, we can even alter the face pose and fix the artifacts accidentally made by GANs. Furthermore, we perform an in-depth face identity analysis and a layer-wise analysis to evaluate the editing results quantitatively. Finally, we apply our approach to real face editing by employing GAN inversion approaches and explicitly training feed-forward models based on the synthetic data established by InterFaceGAN. Extensive experimental results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable face representation.” (see Shen Abstract).
Regarding Independent Claim 23,
Mustikovela discloses a processing system, comprising: a memory having executable instructions stored thereon (see, e.g., paragraph [0111]: “In at least one embodiment, an object detection neural network includes a controllable GAN, an object detection network, and one or more discriminative networks that produce one or more losses… object detection neural network can be implemented with one or more computer systems having one or more processors such as those described below. In at least one embodiment, as a result of executing instructions stored on computer-readable memory, on one or more processors cause a computer system to perform operations described below”);
and a processor configured to execute the executable instructions to cause the processing system to: receive an input data set of scene data; (see, e.g., paragraph [0111]: “In at least one embodiment, as a result of executing instructions stored on computer-readable memory, on one or more processors cause a computer system to perform operations described below” and paragraph [0541]: “In at least one embodiment, a data processing pipeline may receive input data (e.g., imaging data 3708)… In at least one embodiment, input data may be representative of one or more images, video, and/or other data representations generated by one or more imaging devices"):
and map the scene data to a point in one or more multidimensional spaces through a generator model (see, e.g., Mustikovela paragraph [0072]: “In at least one embodiment, S is modeled by a pose-aware generator 202, which synthesizes images {I.sub.g} 204 of objects conditioned on pose parameters 206 (viewpoint (v), location (l), and appearance (z)) and obtain 2D bounding box annotations {A.sub.g} 208 for them”, paragraph [0077]: “input to a network 502, 506 consists of a style vector zf and an object pose in camera coordinates (vf, lf) for one or more foreground objects. In at least one embodiment, value of vf represents azimuth of an object and lf represents horizontal and depth translation. In at least one embodiment, a style vector zb and pose (vb, lf) is input to a second network 504 for background” and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image” [i.e., the localization of an object with depth and pose parameters in a synthesized image is analogous to mapping scene data to points in multidimensional space using synthesis network S (a generator model)]) ;
having a gradient backpropagated to the generator model from one or more critic models5 (see, e.g., Mustikovela paragraph [0073]: “object detection adaptation module is designed to provide feedback to synthesis network S 202 to optimally adapt it to downstream task of object detection… it tightly couples object detector custom-character 302 and synthesizer S 202 for joint end-to-end training and also introduces specific losses 306 and 308 to guide synthesis process towards better object detection learning… a multi-scale discriminator network is used to train the system… a loss generated to train detection network is also used to train a generator network” [i.e., a loss used to provide feedback and adapt the generator functions, is gradient descent being backpropagated from the discriminator network (critic model) to the generator network in the context of training these machine learning models] )
configured to separate points in the one or more multidimensional (see, e.g., Mustikovela paragraph [0078]: “In at least one embodiment, resulting 3D features of objects are collated using an element-wise maximum operation and then projected 514 onto 2D using a perspective camera transformation followed by a set of 2D convolutions 516 to yield I.sub.g 518” [i.e., the projection of 3D features of objects onto 2D during a convolution functions as pushing the points of in the multidimensional space to a plurality of planes in 2D] and paragraph [0081]: “In at least one embodiment, synthesis network S can generate a foreground object using a pose (v.sub.f, l.sub.f). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it. In at least one embodiment, a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (v.sub.f, l.sub.f) via perspective projection” [i.e., the system computes bounding boxes by projecting 3D object coordinates onto a 2D image plane, constraining (separating) the points to 2D plane regions]).
such that the points in the one or more multidimensional spaces are in a vicinity of any of the plurality of planes in the one or more multidimensional spaces (see, e.g., Mustikovela paragraph [0081]: “synthesis network S can generate a foreground object using a pose (vf, lf). In at least one embodiment, this property allows localization of an object in a synthesized image and creation of a 2D bounding box (BBox) annotation for it… a mean 3D bounding box (in real-world dimensions) of an object class is used and projected forward onto a 2D image plane using S's known camera matrix and object's pre-defined pose (vf, lf) via perspective projection… At least one embodiment obtains a 2D bounding box Ag for synthesized image Ig by computing maximum and minimum coordinates of projected 3D bounding box in image plane” [i.e., the coordinates (points) of the projected 3D bounding box in the image plane indicates the points dispersed in the vicinity of image planes in the 3D bounding box latent space]).
Mustikovela do not specifically teach the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces.
Shen teaches the generator model to push the points in the one or more multidimensional spaces to one of a plurality of planes in the one or more multidimensional spaces (see page 3, section 3.1, “Given a well-trained GAN model, the generator can be viewed as a deterministic function g : Z → X. Here, Z ⊆Rd denotes the d-dimensional latent space, for which Gaussian distribution N(0,Id) is commonly used [1], [3], [18], [20]. X stands for the image space, where each sample x possesses certain semantic information, like gender and age for face model. Suppose we have a semantic scoring function fS : X → S, where S ⊆ Rm represents the semantic space with m semantics. We can bridge the latent space Z and the semantic space S with s = fS(g(z)), where s and z denote semantic scores and the sampled latent code respectively.”, also see page 4, section 4, “we apply InterFaceGAN to interpret the face representation learned by state-of-the-art GAN models, i.e., PGGAN [1] and StyleGAN [3], both of which can produce high-quality faces with 1024 × 1024 resolution. PGGAN is a representative of the traditional generator where the latent code is only fed into the very first convolutional layer. By contrast, StyleGAN proposed a style-based generator, which first maps the latent code from latent space Z to a disentangled latent space W before applying it for the generation. In addition, the disentangled latent code is fed to all convolutional layers.”, also see page 6, section 4.2.1, “We surprisingly find that GAN also encodes such quality information in the latent space. Based on this discovery, we manage to correct some mistakes GAN has made in the generation process by moving the latent code towards the positive “quality” direction, as shown in Figure 7.”, also see section 4.2.2).
Both Mustikovela and Shen pertain to the problem of Generative Adversarial Networks (GANs) training, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Mustikovela and Shen to teach the above limitations. The motivation for doing so would be “we propose a framework called InterFaceGAN to interpret the disentangled face representation learned by the state-of-the-art GAN models and study the properties of the facial semantics encoded in the latent space. We first find that GANs learn various semantics in some linear subspaces of the latent space. After identifying these subspaces, we can realistically manipulate the corresponding facial attributes without retraining the model. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection, resulting in more precise control of the attribute manipulation. Besides manipulating the gender, age, expression, and presence of eyeglasses, we can even alter the face pose and fix the artifacts accidentally made by GANs. Furthermore, we perform an in-depth face identity analysis and a layer-wise analysis to evaluate the editing results quantitatively. Finally, we apply our approach to real face editing by employing GAN inversion approaches and explicitly training feed-forward models based on the synthetic data established by InterFaceGAN. Extensive experimental results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable face representation.” (see Shen Abstract).
Regarding claim 24, as discussed above Mustikovela and Shen teaches the system of claim 23. Mustikovela further discloses wherein the processor is further configured to cause the processing system to predict a location on a plane in the one or more multidimensional spaces (see, e.g., paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose as input and generates an image used to train object detection neural network. In at least one embodiment, a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose” [i.e., the object detection model trains an object detection model that calculates a loss and pose (spatial information in a geometric space) for the location of objects (predicting the location of an objects) in an image which is a prediction on a 2D plane])
at which the received scene data is located based on the point in the one or more multidimensional spaces to which the scene data is mapped (see, e.g., paragraph [0151]: “In at least one embodiment, a generative adversarial network takes an object location and pose… a loss is determined based at least in part on a difference between output of object detection neural network and an a specified object location and pose” [i.e., the predicted object location/pose at a point in latent space is directly derived from the objects location and pose input (scene data) in the 2D image plane]).
Regarding claim 25, above Mustikovela and Shen teaches the system of claim 23. Mustikovela further discloses wherein the scene data comprises one or more images of a spatial environment (see, e.g., paragraph(s) [0071]: “In at least one embodiment, a goal is to learn a detection network Figure ‘F’, which best detects objects (such as cars) in a target domain (such as out-door driving scenes from a city). At least one embodiment further assumes availability of an unlabeled image collection {It} from target domain each containing an unknown number of objects per image (see FIG. 1 ). To train Figure ‘F’, at least one embodiment leverages object images and their bounding box annotations synthesized by a controllable generative network S 202" [i.e., the target domain out-door driving scenes from a city includes images from a spatial environment] ).
Claims 3, 13 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela (US 20230004760 A1) in view of Shan et al. (“InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs”, 2020) in view of Doken (US 20230091437 A1; hereinafter Doken).
Regarding claim 3, as discussed above, Mustikovela and Shen teaches the method of claim 1. However, Mustikovela fails to explicitly teach wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time.
Nevertheless, in the same field, analogous art Doken teaches wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time (see, e.g., Doken paragraph [0047] “the wireless signal-sensing application may generate map 200 and use map 200 to determine what the wireless environment of the household looks like, and the wireless signal-sensing application may identify subsequent changes in that landscape, e.g., based on the way the RF energy is absorbed or reflected, which may correlate with someone or something being present or not present, and/or moving or being stationary. In some embodiments, a time series of wireless signal information (e.g., CSI, RSSI, RCPI) may be collected by router” [i.e., RCPI (received channel power indicator) is measured in the generated map providing power density information associated with CSI measurements obtained in the household environment (scene data) over time]).
Mustikovela, Shen and Doken are analogous art because they are both directed to the machine learning classification (see, e.g., Mustikovela, paragraph [0130] and Doken, paragraph [0012]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Doken to utilize scene data that includes a power density map associated with channel state information (CSI) measurements obtained over time. Doing so would have allowed Mustikovela to use Doken's method in order to “determine, without requiring user input, whether a user has moved out of range of a consumption device”, as suggested by Doken (see, e.g., Doken, paragraph [0006]).
Regarding claim 13, as discussed above, Mustikovela and Shen teaches the method of claim 10. However, Mustikovela fails to explicitly teach wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time.
Nevertheless, in the same field, analogous art Doken teaches wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time (see, e.g., Doken paragraph [0047] “the wireless signal-sensing application may generate map 200 and use map 200 to determine what the wireless environment of the household looks like, and the wireless signal-sensing application may identify subsequent changes in that landscape, e.g., based on the way the RF energy is absorbed or reflected, which may correlate with someone or something being present or not present, and/or moving or being stationary. In some embodiments, a time series of wireless signal information (e.g., CSI, RSSI, RCPI) may be collected by router” [i.e., RCPI (received channel power indicator) is measured in the generated map providing power density information associated with CSI measurements obtained in the household environment (scene data) over time]).
Mustikovela, Shen and Doken are analogous art because they are both directed to the machine learning classification (see, e.g., Mustikovela, paragraph [0130] and Doken, paragraph [0012]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Doken to utilize scene data that includes a power density map associated with channel state information (CSI) measurements obtained over time. Doing so would have allowed Mustikovela to use Doken's method in order to “determine, without requiring user input, whether a user has moved out of range of a consumption device”, as suggested by Doken (see, e.g., Doken, paragraph [0006]).
Regarding claim 26, as discussed above, Mustikovela and Shen teaches the system of claim 23. However, Mustikovela and Shen fails to explicitly teach wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time.
Nevertheless, in the same field, analogous art Doken teaches wherein the scene data comprises a power density map associated with channel state information (CSI) measurements obtained over time (see, e.g., Doken paragraph [0047] “the wireless signal-sensing application may generate map 200 and use map 200 to determine what the wireless environment of the household looks like, and the wireless signal-sensing application may identify subsequent changes in that landscape, e.g., based on the way the RF energy is absorbed or reflected, which may correlate with someone or something being present or not present, and/or moving or being stationary. In some embodiments, a time series of wireless signal information (e.g., CSI, RSSI, RCPI) may be collected by router” [i.e., RCPI (received channel power indicator) is measured in the generated map providing power density information associated with CSI measurements obtained in the household environment (scene data) over time]).
Mustikovela, Shen and Doken are analogous art because they are both directed to the machine learning classification (see, e.g., Mustikovela, paragraph [0130] and Doken, paragraph [0012]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Doken to utilize scene data that includes a power density map associated with channel state information (CSI) measurements obtained over time. Doing so would have allowed Mustikovela to use Doken's method in order to “determine, without requiring user input, whether a user has moved out of range of a consumption device”, as suggested by Doken (see, e.g., Doken, paragraph [0006]).
17. Claims 4, 19 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela and Shen, as applied to claims 1 and 18 above, in view of Shechtman (US 20190251401 A1; hereinafter Shechtman).
Regarding claim 4, Mustikovela teaches and a second critic model configured to enforce granularity among outputs of the generator model (see, e.g., Mustikovela paragraph [0073]: “In at least one embodiment, a multi-scale discriminator network is used to train the system” [i.e., a multi-scale discriminator network acts as a second critic model] and paragraph [0084]: “to extend range of depths for which S generates high-quality objects, a multi-scale object synthesis loss is introduced… At least one embodiment resizes l.sub.c to 256×256. In at least one embodiment, this creates effect of magnifying/zooming into smaller regions of synthesized images. In at least one embodiment, to ensure that a high-quality car is indeed synthesized faithfully in smaller l.sub.c window… I.sub.c is passed, to a multi-scale object discriminator” [i.e., the multi-scale object discriminator (second critic model) is fed magnified synthesized images for scaling the images (enforcing the granularity) among outputs of the generator model S])
However, Mustikovela and Shen fails to explicitly teach the limitation wherein the one or more critic models comprise a first critic model configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model.
Nevertheless, in the same field, analogous art Shechtman teaches wherein the one or more critic models comprise a first critic model configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model (see, e.g., Shechtman paragraphs [0042-0044] “As used herein, the term “warp parameters” refers to a homograph transformation matrix that stores numerical values on how a given image (e.g., a foreground object image) is spatially transformed to change the perspective, orientation, size, position, and/or geometry of the object to better conform to a background image. In one or more embodiments, warp parameters are restricted to approximate geometric rectifications for objects that are mostly planar (e.g., in-planer modifications) or with small perturbations… As mentioned above, the image composite system can combine the geometric prediction neural network and the adversarial discrimination neural network to form a generative adversarial network (GAN)” and paragraph [0052]: “the feedback 118 indicates to the geometric prediction neural network 102 whether the warp parameters 110 fooled the adversarial discrimination neural network 104 into classifying the composite image 110 as a real image. If the geometric prediction neural network 102 did not fool the adversarial discrimination neural network 104, the image composite system can iteratively update the weights and parameters of the geometric prediction neural network 102 to determine more realistic warp parameters”
[i.e., the adversarial discrimination neural network 104 (critic model) promotes realistic warp parameters, a geometric constraint to achieve co-planarity of points in the homographic transformation] and paragraph [0088]: “To illustrate, FIG. 3 shows a diagram of employing a trained generative adversarial network 300 that generates realistic composite images” [i.e., the GAN 300 (generator model) generates composite images (multidimensional spaces, i.e., layers of color channels and details)]).
Mustikovela, Shen and Shechtman are analogous art because they are both directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Shechtman, paragraph [0027]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Shechtman to use a critic model to promote co-planarity of points in the one or more multidimensional spaces generated by a generator model. Doing so would have allowed Mustikovela to use Shechtman's method in order to “to generate warp parameters that provide correct geometric alignment of foreground objects with respect to a background image”, as suggested by Shechtman (see, e.g., Shechtman, paragraph [0005]).
Regarding claim 19, Mustikovela and Shen teaches the system of claim 10. Mustikovela further teaches and a second critic model6 configured to enforce granularity among outputs of the generator model (see, e.g., Mustikovela paragraph [0073]: “In at least one embodiment, a multi-scale discriminator network is used to train the system” [i.e., a multi-scale discriminator network acts as a second critic model] and paragraph [0084]: “to extend range of depths for which S generates high-quality objects, a multi-scale object synthesis loss is introduced… At least one embodiment resizes l.sub.c to 256×256. In at least one embodiment, this creates effect of magnifying/zooming into smaller regions of synthesized images. In at least one embodiment, to ensure that a high-quality car is indeed synthesized faithfully in smaller l.sub.c window… I.sub.c is passed, to a multi-scale object discriminator” [i.e., the multi-scale object discriminator (second critic model) is fed magnified synthesized images for scaling the images (enforcing the granularity) among outputs of the generator model S])
Although Mustikovela thoroughly teaches the claimed invention, Mustikovela and Shen teaches fails to explicitly teach the limitation wherein the one or more critic models comprise a first critic model7 configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model
Nevertheless, in the same field, analogous art Shechtman teaches wherein the one or more critic models comprise a first critic model configured to promote co-planarity of points in the one or more multidimensional spaces generated by the generator model (see, e.g., Shechtman paragraph [0042-0044] “As used herein, the term “warp parameters” refers to a homograph transformation matrix that stores numerical values on how a given image (e.g., a foreground object image) is spatially transformed to change the perspective, orientation, size, position, and/or geometry of the object to better conform to a background image. In one or more embodiments, warp parameters are restricted to approximate geometric rectifications for objects that are mostly planar (e.g., in-planer modifications) or with small perturbations… As mentioned above, the image composite system can combine the geometric prediction neural network and the adversarial discrimination neural network to form a generative adversarial network (GAN)” and paragraph [0052]: “the feedback 118 indicates to the geometric prediction neural network 102 whether the warp parameters 110 fooled the adversarial discrimination neural network 104 into classifying the composite image 110 as a real image. If the geometric prediction neural network 102 did not fool the adversarial discrimination neural network 104, the image composite system can iteratively update the weights and parameters of the geometric prediction neural network 102 to determine more realistic warp parameters”
[i.e., the adversarial discrimination neural network 104 (critic model) promotes realistic warp parameters, a geometric constraint to achieve co-planarity of points in the homographic transformation] and paragraph [0088]: “To illustrate, FIG. 3 shows a diagram of employing a trained generative adversarial network 300 that generates realistic composite images” [i.e., the GAN 300 (generator model) generates composite images (multidimensional spaces, i.e., layers of color channels and details)]).
Mustikovela, Shen and Shechtman are analogous art because they are both directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Shechtman, paragraph [0027]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Shechtman to use a critic model to promote co-planarity of points in the one or more multidimensional spaces generated by a generator model. Doing so would have allowed Mustikovela to use Shechtman's method in order to “to generate warp parameters that provide correct geometric alignment of foreground objects with respect to a background image”, as suggested by Shechtman (see, e.g., Shechtman, paragraph [0005]).
Regarding claim 31. Mustikovela teaches the method of Claim 1, However, Mustikovela do not specifically teach wherein a first plane of the plurality of planes is located above a second plane of the plurality of planes. Nevertheless, in the same field, analogous art Shechtman teaches wherein a first plane of the plurality of planes is located above a second plane of the plurality of planes (see, e.g., Shechtman paragraphs [0042-0044] “As used herein, the term “warp parameters” refers to a homograph transformation matrix that stores numerical values on how a given image (e.g., a foreground object image) is spatially transformed to change the perspective, orientation, size, position, and/or geometry of the object to better conform to a background image. In one or more embodiments, warp parameters are restricted to approximate geometric rectifications for objects that are mostly planar (e.g., in-planer modifications) or with small perturbations… As mentioned above, the image composite system can combine the geometric prediction neural network and the adversarial discrimination neural network to form a generative adversarial network (GAN)” and paragraph [0052]: “the feedback 118 indicates to the geometric prediction neural network 102 whether the warp parameters 110 fooled the adversarial discrimination neural network 104 into classifying the composite image 110 as a real image. If the geometric prediction neural network 102 did not fool the adversarial discrimination neural network 104, the image composite system can iteratively update the weights and parameters of the geometric prediction neural network 102 to determine more realistic warp parameters”
[i.e., the adversarial discrimination neural network 104 (critic model) promotes realistic warp parameters, a geometric constraint to achieve co-planarity of points in the homographic transformation] and paragraph [0088]: “To illustrate, FIG. 3 shows a diagram of employing a trained generative adversarial network 300 that generates realistic composite images” [i.e., the GAN 300 (generator model) generates composite images (multidimensional spaces, i.e., layers of color channels and details)]).
Mustikovela, Shen and Shechtman are analogous art because they are both directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Shechtman, paragraph [0027]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Shechtman to use a critic model to promote co-planarity of points in the one or more multidimensional spaces generated by a generator model. Doing so would have allowed Mustikovela to use Shechtman's method in order to “to generate warp parameters that provide correct geometric alignment of foreground objects with respect to a background image”, as suggested by Shechtman (see, e.g., Shechtman, paragraph [0005]).
18. Claims 5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela (US 20230004760 A1) in view of Shan et al. (“InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs”, 2020) in view of Shechtman, as applied to claims 4 and 19 above, and further in view of Miyazawa (US 20230326052 A1; hereinafter Miyazawa).
Regarding Claim 5, as discussed above, Mustikovela and Shen in view of Shechtman teaches the method of claim 4.
Although Mustikovela and Shen in view of Shechtman substantially teaches the claimed invention, Mustikovela and Shen in view of Shechtman fails to explicitly teach wherein the first critic model is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane
in the one or more multidimensional spaces for an input of scene data
In the same field, analogous art Miyazawa teaches wherein the first critic model is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane (see, e.g., Miyazawa paragraph [0023]: “The plane estimation unit 17 estimates parameters of the plane that fits the coordinates of the point clouds including the point cloud whose coordinates have been shifted. Here, the plane estimation unit 17 executes the plane fitting processing using a predetermined algorithm. The predetermined algorithm is, for example, RANSAC. The plane estimation unit 17 may execute plane fitting processing using a deep learning network of a differentiable RANSAC disclosed in Reference 3 (Brachmann, Eric, et al. ‘Dsac-differentiable ransac for camera localization.’ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.). The plane estimation unit 17 learns all of processing including the plane fitting processing by using the deep learning network of the differentiable RANSAC. Accordingly, because the plane fitting processing optimized for the output of the shift processing unit 16 is executed, improvement in the estimation accuracy of the parameters of the plane can be expected” [i.e., the deep learning network of a differentiable RANSAC plane fitting process functions as a critic model that selects the best hypothesis of a plane based on the number or inliers along the plane (see, e.g., Introduction to Brachmann, Eric, et al. ‘Dsac-differentiable ransac for camera localization.’ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.)]).
in the one or more multidimensional spaces for an input of scene data (see, e.g., Miyazawa paragraph [0023]: “Accordingly, the depth sensor generates point cloud data of the section of the room. The point cloud data is three-dimensional coordinates of a point cloud. When the estimation device 1 estimates a shape of the section of the room, the point cloud data may be a depth map” [i.e., the hypothesis of best selection of a plane is used for 3D room selection (multidimensional spaces of input scene data)]).
Mustikovela, shen, Shechtman and Miyazawa are analogous art because they are each directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074], Shechtman, paragraph [0027], and Miyazawa [0049]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela in view of Shechtman to incorporate the teachings of Mayazawa to use a critic model configured to select a best hypothesis of a plane in one or more multidimensional spaces based on a number of inliers along each plane in one or more multidimensional spaces for an input of scene data. Doing so would have allowed Mustikovela in view of Shechtman to use Mayazawa's method in order to "provide an estimation method… capable of estimating a shape of a section without using an RGB image when a part of the section cannot be seen due to disposition of an obstacle”, as suggested by Miyazawa (see, e.g., Miyazawa, paragraph [0011]).
Regarding Claim 20, as discussed above, Mustikovela and Shen in view of Shechtman teaches the method of claim 19.
Although Mustikovela and Shen in view of Shechtman substantially teaches the claimed invention, Mustikovela in view of Shechtman fails to explicitly teach wherein the first critic model8 is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane
in the one or more multidimensional spaces for an input of scene data
In the same field, analogous art Miyazawa teaches wherein the first critic model is configured to select a best hypothesis of a plane in the one or more multidimensional spaces based on a number of inliers along each plane (see, e.g., Miyazawa paragraph [0023]: “The plane estimation unit 17 estimates parameters of the plane that fits the coordinates of the point clouds including the point cloud whose coordinates have been shifted. Here, the plane estimation unit 17 executes the plane fitting processing using a predetermined algorithm. The predetermined algorithm is, for example, RANSAC. The plane estimation unit 17 may execute plane fitting processing using a deep learning network of a differentiable RANSAC disclosed in Reference 3 (Brachmann, Eric, et al. ‘Dsac-differentiable ransac for camera localization.’ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.). The plane estimation unit 17 learns all of processing including the plane fitting processing by using the deep learning network of the differentiable RANSAC. Accordingly, because the plane fitting processing optimized for the output of the shift processing unit 16 is executed, improvement in the estimation accuracy of the parameters of the plane can be expected” [i.e., the deep learning network of a differentiable RANSAC plane fitting process functions as a critic model that selects the best hypothesis of a plane based on the number or inliers along the plane (see, e.g., Introduction to Brachmann, Eric, et al. ‘Dsac-differentiable ransac for camera localization.’ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.)]).
in the one or more multidimensional spaces for an input of scene data (see, e.g., Miyazawa paragraph [0023]: “Accordingly, the depth sensor generates point cloud data of the section of the room. The point cloud data is three-dimensional coordinates of a point cloud. When the estimation device 1 estimates a shape of the section of the room, the point cloud data may be a depth map” [i.e., the hypothesis of best selection of a plane is used for 3D room selection (multidimensional spaces of input scene data)]).
Mustikovela, shen, Shechtman and Miyazawa are analogous art because they are each directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074], Shechtman, paragraph [0027], and Miyazawa [0049]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela in view of Shechtman to incorporate the teachings of Mayazawa to use a critic model configured to select a best hypothesis of a plane in one or more multidimensional spaces based on a number of inliers along each plane in one or more multidimensional spaces for an input of scene data. Doing so would have allowed Mustikovela in view of Shechtman to use Mayazawa's method in order to “ provide an estimation method… capable of estimating a shape of a section without using an RGB image when a part of the section cannot be seen due to disposition of an obstacle”, as suggested by Miyazawa (see, e.g., Miyazawa, paragraph [0011]).
19. Claims 6, 7 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela (US 20230004760 A1) in view of Shan et al. (“InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs”, 2020) in view of Shechtman and further in view of Phalak (US 20210279950A1; hereinafter Phalak).
Regarding Claim 6, as discussed above, Mustikovela and Shen in view of Shechtman teaches the method of claim 4.
Although Mustikovela and Shen in view of Shechtman substantially teaches the claimed invention, Mustikovela and Shen in view of Shechtman fails to explicitly teach wherein the first critic model generates an attraction force and a repulsion force to apply to mapped data in the one or more multidimensional spaces
In the same field, analogous art Phalak teaches wherein the first critic model generates an attraction force and a repulsion force to apply to mapped data in the one or more multidimensional spaces (see, e.g., Phalak paragraph [0383-0384]: “D(x i ,x j)=(x j −x j)x i (n)+(x j −x i)x i (n), Eq. 15C-(3) The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane… Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., the same plane point yields a low penalty keeping the points on the plane, acting as an attraction force, and points on different planes are penalized with a greater term when they are clustered together, acting as a repulsion force that pushes the values apart, in the context of 3D walls in multidimensional space]”).
Mustikovela, shen, Shechtman and Phalak are analogous art because they are each directed to machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074], Shechtman, paragraph [0027], and Phalak [0156]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela in view of Shechtman to incorporate the teachings of Phalak to use a first critic model that generates an attraction force and a repulsion force to apply to mapped data in one or more multidimensional spaces . Doing so would have allowed Mustikovela in view of Shechtman to use Phalak's method for “efficient generation of a floorplan from scans of indoor scenes to address at least the aforementioned deficiencies, challenges, shortcomings, and difficulties of conventional approaches”, as suggested by Phalak (see, e.g., Phalak, paragraph [0014]).
Regarding Claim 7, as discussed above, Mustikovela and Shen in view of Shechtman teaches the method of claim 4.
Although Mustikovela and Shen in view of Shechtman substantially teaches the claimed invention, Mustikovela and Shen in view of Shechtman fails to explicitly teach wherein the second critic model is configured to minimize a loss between adjacent clusters in a spatial environment
In the same field, analogous art Phalak teaches wherein the second critic model is configured to minimize a loss between adjacent clusters in a spatial environment (see, e.g., Phalak paragraph [0381]: “The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” and paragraph [0384]: “[0384] The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane… Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., the loss function penalizes (seeks to minimize) similar (adjacent) clustering of pairs belonging to distinct walls]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela in view of Shechtman to incorporate the teachings of Phalak to have the second critic model of Mustikovela minimize a loss between adjacent clusters in a spatial environment. Doing so would have allowed Mustikovela in view of Shechtman to use Phalak's method for “efficient generation of a floorplan from scans of indoor scenes to address at least the aforementioned deficiencies, challenges, shortcomings, and difficulties of conventional approaches”, as suggested by Phalak (see, e.g., Phalak, paragraph [0014]).
Regarding Claim 21, as discussed above, Mustikovela and Shen in view of Shechtman teaches the system of claim 19.
Although Mustikovela and Shen in view of Shechtman substantially teaches the claimed invention, Mustikovela in view of Shechtman fails to explicitly teach wherein the second critic model9 is configured to minimize a loss between adjacent clusters in a spatial environment
In the same field, analogous art Phalak teaches wherein the second critic model is configured to minimize a loss between adjacent clusters in a spatial environment (see, e.g., Phalak paragraph [0381]: “The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” and paragraph [0384]: “The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane… Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., the loss function penalizes (seeks to minimize) similar (adjacent) clustering of pairs belonging to distinct walls]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela in view of Shechtman to incorporate the teachings of Phalak to have the second critic model of Mustikovela minimize a loss between adjacent clusters in a spatial environment. Doing so would have allowed Mustikovela in view of Shechtman to use Phalak's method for “efficient generation of a floorplan from scans of indoor scenes to address at least the aforementioned deficiencies, challenges, shortcomings, and difficulties of conventional approaches”, as suggested by Phalak (see, e.g., Phalak, paragraph [0014]).
20. Claims 14, 15, 27 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela and Shen in view of Phalak (US 20210279950A1; hereinafter Phalak).
Regarding claim 14, as discussed above, Mustikovela and Shen teaches the method of claim 10. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane.
Nevertheless, in the same field, analogous art Phalak teaches wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane (see, e.g., Mustikovela paragraph [0380-0381]: “ClusterNet: Upon obtaining an a-culled, subsampled point cloud representation of the walls in the scene, some embodiments proceed to separate the wall instances by a performing a deep clustering of this point cloud. Some embodiments employ a fully unsupervised technique of clustering unordered point clouds based on planar sections… Furthermore, the clustering needs to distinguish between parallel planar walls which share the same point normals. Some embodiments formulate a pairwise loss function that penalizes the network when two points lying on distinct wall instances are assigned the same label… The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” [i.e., the loss function in the ClusterNet implies a gradient used to train the neural network to perform the clustering technique] and paragraphs [0383-0384]: “and D(x i ,x j)=(x j −x j)x i (n)+(x j −x i)x i (n), Eq. 15C-(3)… The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane” [i.e., the same plane point yields a low penalty keeping the points on the plane, acting as an attraction force]).
and a repulsion force that pushes points located on different planes in the one or more multidimensional spaces away from each other (see, e.g., Phalak paragraphs [0383-0384]: “and D(x i ,x j)=(x j −x j)x i (n)+(x j −x i)x i (n), Eq. 15C-(3)… The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) … Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., points on different planes are penalized with a greater term when they are clustered together, acting as a repulsion force that pushes the values apart]).
Mustikovela, Shen and Phalak are analogous art because they are both directed to machine learning techniques for multi-dimensional environments (see, e.g., Mustikovela, paragraph [0081], Phalak, paragraphs [0284-0285]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Phalak to incorporate a gradient that includes an attraction force term that pushes co-planar points in one or more multidimensional spaces onto a same plane and a repulsion force that pushes points located on different planes in one or more multidimensional spaces away from each other. Doing so would have allowed Mustikovela to use Phalak's method in order to “divide the set of points into a plurality of overlapping local regions based at least in part upon a distance metric pertaining to the indoor scene”, as suggested by Phalak (see, e.g., Phalak, paragraph [0018]).
Regarding claim 15, as discussed above, Mustikovela and Shen teaches the system of claim 10. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment.
Nevertheless, in the same field, analogous art Phalak teaches wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment (see, e.g., Phalak paragraphs [0381] “Furthermore, the clustering needs to distinguish between parallel planar walls which share the same point normals. Some embodiments formulate a pairwise loss function that penalizes the network when two points lying on distinct wall instances are assigned the same label… The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” [i.e., the loss function in the ClusterNet implies a gradient used to train the neural network to perform the clustering technique to points of walls (in a spatial environment)] and paragraph [0384]: “The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane… Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., the loss function penalizes (seeks to minimize) similar (adjacent) clustering of pairs belonging to distinct walls]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Phalak to incorporate a gradient with a minimized loss between adjacent clusters in a spatial environment. Doing so would have allowed Mustikovela to use Phalak's method in order to “determine the room classification of the room and the wall classification of the wall” [i.e., the classification of different planes in a spatial environment], as suggested by Phalak (see, e.g., Phalak, paragraph [0018]).
Regarding claim 27, as discussed above, Mustikovela and Shen teaches the system of claim 23. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane.
Nevertheless, in the same field, analogous art Phalak teaches wherein the gradient comprises an attraction force term that pushes co-planar points in the one or more multidimensional spaces onto a same plane (see, e.g., Mustikovela paragraph [0380-0381]: “ClusterNet: Upon obtaining an a-culled, subsampled point cloud representation of the walls in the scene, some embodiments proceed to separate the wall instances by a performing a deep clustering of this point cloud. Some embodiments employ a fully unsupervised technique of clustering unordered point clouds based on planar sections… Furthermore, the clustering needs to distinguish between parallel planar walls which share the same point normals. Some embodiments formulate a pairwise loss function that penalizes the network when two points lying on distinct wall instances are assigned the same label… The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” [i.e., the loss function in the ClusterNet implies a gradient used to train the neural network to perform the clustering technique] and paragraphs [0383-0384]: “and D(x i ,x j)=(x j −x j)x i (n)+(x j −x i)x i (n), Eq. 15C-(3)… The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane” [i.e., the same plane point yields a low penalty keeping the points on the plane, acting as an attraction force]).
and a repulsion force that pushes points located on different planes in the one or more multidimensional spaces away from each other (see, e.g., Phalak paragraphs [0383-0384]: “and D(x i ,x j)=(x j −x j)x i (n)+(x j −x i)x i (n), Eq. 15C-(3)… The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) … Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., points on different planes are penalized with a greater term when they are clustered together, acting as a repulsion force that pushes the values apart]).
Mustikovela, Shen and Phalak are analogous art because they are both directed to machine learning techniques for multi-dimensional environments (see, e.g., Mustikovela, paragraph [0081], Phalak, paragraphs [0284-0285]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Phalak to incorporate a gradient that includes an attraction force term that pushes co-planar points in one or more multidimensional spaces onto a same plane and a repulsion force that pushes points located on different planes in one or more multidimensional spaces away from each other. Doing so would have allowed Mustikovela to use Phalak's method in order to “divide the set of points into a plurality of overlapping local regions based at least in part upon a distance metric pertaining to the indoor scene”, as suggested by Phalak (see, e.g., Phalak, paragraph [0018]).
Regarding claim 28, as discussed above, Mustikovela and Shen teaches discloses the system of claim 23. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment.
Nevertheless, in the same field, analogous art Phalak teaches wherein the gradient comprises a minimized loss between adjacent clusters in a spatial environment (see, e.g., Phalak paragraphs [0381] “Furthermore, the clustering needs to distinguish between parallel planar walls which share the same point normals. Some embodiments formulate a pairwise loss function that penalizes the network when two points lying on distinct wall instances are assigned the same label… The clustering loss Lcluster is given as: L cluster=Σi,j>i P(x i ,x j)D(x i ,j i), Eq. 15C-(1)” [i.e., the loss function in the ClusterNet implies a gradient used to train the neural network to perform the clustering technique to points of walls (in a spatial environment)] and paragraph [0384]: “The term D(xi, xj) has a high value when xi and xj lie on different planes (including parallel planes) and a low value when xi and xj lie on the same plane… Some embodiments penalize when two pairs belong to distinct walls but have a similar cluster assignment probability” [i.e., the loss function penalizes (seeks to minimize) similar (adjacent) clustering of pairs belonging to distinct walls]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Phalak to incorporate a gradient with a minimized loss between adjacent clusters in a spatial environment. Doing so would have allowed Mustikovela to use Phalak's method in order to “determine the room classification of the room and the wall classification of the wall” [i.e., the classification of different planes in a spatial environment], as suggested by Phalak (see, e.g., Phalak, paragraph [0018]).
Claims 8, 16, 22, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela (US 20230004760 A1) in view of Shan et al. (“InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs”, 2020) in view of Chi (US 20210279840 A1; hereinafter Chi).
Regarding claim 8, as discussed above, Mustikovela and Shen teaches the method of claim 18. However, Mustikovela and Shen fails to explicitly teach further comprising training the generator model and the one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Nevertheless, in the same field, analogous art Chi teaches further comprising training the generator model and the one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time (see, e.g., Chi paragraph [0026]: "Some embodiments use higher-order motion modeling than existing multiple-frame video frame interpolation approaches, such as cubic motion modeling (as opposed to the quadratic or linear motion modeling used by existing approaches), to achieve more accurate predictions of intermediate optical flow between multiple interpolated new intermediate frames between a beginning frame and an ending frame of a sequence of frames of a digital video" [i.e., the cubic motion modeling equation is used for ensuring temporal consistency in interpolated frames functioning as a scene flow constraint term], "Some embodiments use a temporal pyramidal optical flow refinement module to perform coarse-to-fine refinement of the optical flow maps used to generate (e.g. interpolate) new intermediate frames between a beginning frame and an ending frame of a video sequence, focusing a proportionally greater amount of refinement attention to the optical flow maps for the high-error middle-most frames (i.e. the intermediate frames having a timestamp closest to the temporal midpoint between the beginning frame timestamp and the ending frame timestamp)” [i.e., the optical flow frames in the middle-most-frames have the highest expected errors due to their greater temporal distance between the beginning and ending frame timestamps, meaning the constraint term increases with a temporal offset these initial points in time] and “Some embodiments use a module which implements a generative adversarial network (GAN) to compute a loss for training of the neural networks implemented in the optical flow estimation module, temporal pyramidal optical flow refinement module, and/or temporal pyramidal pixel refinement module” [i.e., the computation of a loss is used for training of the GAN neural networks using the scene flow constraints, which guides the optical flow refinement module to generate accurate predictions of intermediate optical flow between video frames]).
Mustikovela Shen and Chi are analogous art because they are both directed machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Chi [0026]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Chi to train a generator model and one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time. Doing so would have allowed Mustikovela to use Chi's method in order to “consider temporal consistency by applying adaptive processing to the optical flow maps used to generate the intermediate frames, and/or to the generated intermediate frames themselves, to focus processing on the high-error middle-most frames”, as suggested by Chi (see, e.g., Chi, paragraph [0027]).
Regarding claim 16, as discussed above, Mustikovela and Shen teaches the method of claim 10. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Nevertheless, in the same field, analogous art Chi teaches wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time (see, e.g., Chi paragraph [0026]: "Some embodiments use higher-order motion modeling than existing multiple-frame video frame interpolation approaches, such as cubic motion modeling (as opposed to the quadratic or linear motion modeling used by existing approaches), to achieve more accurate predictions of intermediate optical flow between multiple interpolated new intermediate frames between a beginning frame and an ending frame of a sequence of frames of a digital video…" [i.e., the cubic motion modeling equation is used for ensuring temporal consistency in interpolated frames functioning as a scene flow constraint term], "Some embodiments use a temporal pyramidal optical flow refinement module to perform coarse-to-fine refinement of the optical flow maps used to generate (e.g. interpolate) new intermediate frames between a beginning frame and an ending frame of a video sequence, focusing a proportionally greater amount of refinement attention to the optical flow maps for the high-error middle-most frames (i.e. the intermediate frames having a timestamp closest to the temporal midpoint between the beginning frame timestamp and the ending frame timestamp)” [i.e., the optical flow frames in the middle-most-frames have the highest expected errors due to their greater temporal distance between the beginning and ending frame timestamps, meaning the constraint term increases with a temporal offset these initial points in time] and “Some embodiments use a module which implements a generative adversarial network (GAN) to compute a loss for training of the neural networks implemented in the optical flow estimation module, temporal pyramidal optical flow refinement module, and/or temporal pyramidal pixel refinement module” [i.e., the computation of a loss for training of the GAN neural networks using the constraints implies a gradient is will be produced for optimizing the optical flow frame predictions]).
Mustikovela Shen and Chi are analogous art because they are both directed machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Chi [0026]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Chi to use a gradient that has a scene flow constraint term that increases in value with a temporal offset from an initial point in time. Doing so would have allowed Mustikovela to use Chi's method in order to “consider temporal consistency by applying adaptive processing to the optical flow maps used to generate the intermediate frames, and/or to the generated intermediate frames themselves, to focus processing on the high-error middle-most frames”, as suggested by Chi (see, e.g., Chi, paragraph [0027]).
Regarding claim 22, as discussed above, Mustikovela and Shen teaches the system of claim 18. However, Mustikovela and Shen fails to explicitly teach wherein the processor is further configured to cause the processing system to train the generator model and the one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time.
Nevertheless, in the same field, analogous art Chi teaches wherein the processor is further configured to cause the processing system to train the generator model and the one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time (see, e.g., Chi paragraph [0026]: "Some embodiments use higher-order motion modeling than existing multiple-frame video frame interpolation approaches, such as cubic motion modeling (as opposed to the quadratic or linear motion modeling used by existing approaches), to achieve more accurate predictions of intermediate optical flow between multiple interpolated new intermediate frames between a beginning frame and an ending frame of a sequence of frames of a digital video…" [i.e., the cubic motion modeling equation is used for ensuring temporal consistency in interpolated frames functioning as a scene flow constraint term], "Some embodiments use a temporal pyramidal optical flow refinement module to perform coarse-to-fine refinement of the optical flow maps used to generate (e.g. interpolate) new intermediate frames between a beginning frame and an ending frame of a video sequence, focusing a proportionally greater amount of refinement attention to the optical flow maps for the high-error middle-most frames (i.e. the intermediate frames having a timestamp closest to the temporal midpoint between the beginning frame timestamp and the ending frame timestamp)” [i.e., the optical flow frames in the middle-most-frames have the highest expected errors due to their greater temporal distance between the beginning and ending frame timestamps, meaning the constraint term increases with a temporal offset these initial points in time] and “Some embodiments use a module which implements a generative adversarial network (GAN) to compute a loss for training of the neural networks implemented in the optical flow estimation module, temporal pyramidal optical flow refinement module, and/or temporal pyramidal pixel refinement module” [i.e., the computation of a loss is used for training of the GAN neural networks using the scene flow constraints, which guides the optical flow refinement module to generate accurate predictions of intermediate optical flow between video frames]).).
Mustikovela Shen and Chi are analogous art because they are both directed machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Chi [0026]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Chi to train a generator model and one or more critic models based on a scene flow constraint term that increases in value with a temporal offset from an initial point in time. Doing so would have allowed Mustikovela to use Chi's method in order to “consider temporal consistency by applying adaptive processing to the optical flow maps used to generate the intermediate frames, and/or to the generated intermediate frames themselves, to focus processing on the high-error middle-most frames”, as suggested by Chi (see, e.g., Chi, paragraph [0027]).
Regarding claim 29, as discussed above, Mustikovela and Shen teaches the system of claim 23. However, Mustikovela and Shen fails to explicitly teach wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time
Nevertheless, in the same field, analogous art Chi teaches wherein the gradient comprises a scene flow constraint term that increases in value with a temporal offset from an initial point in time (see, e.g., Chi paragraph [0026]: "Some embodiments use higher-order motion modeling than existing multiple-frame video frame interpolation approaches, such as cubic motion modeling (as opposed to the quadratic or linear motion modeling used by existing approaches), to achieve more accurate predictions of intermediate optical flow between multiple interpolated new intermediate frames between a beginning frame and an ending frame of a sequence of frames of a digital video…" [i.e., the cubic motion modeling equation is used for ensuring temporal consistency in interpolated frames functioning as a scene flow constraint term], "Some embodiments use a temporal pyramidal optical flow refinement module to perform coarse-to-fine refinement of the optical flow maps used to generate (e.g. interpolate) new intermediate frames between a beginning frame and an ending frame of a video sequence, focusing a proportionally greater amount of refinement attention to the optical flow maps for the high-error middle-most frames (i.e. the intermediate frames having a timestamp closest to the temporal midpoint between the beginning frame timestamp and the ending frame timestamp)” [i.e., the optical flow frames in the middle-most-frames have the highest expected errors due to their greater temporal distance between the beginning and ending frame timestamps, meaning the constraint term increases with a temporal offset these initial points in time] and “Some embodiments use a module which implements a generative adversarial network (GAN) to compute a loss for training of the neural networks implemented in the optical flow estimation module, temporal pyramidal optical flow refinement module, and/or temporal pyramidal pixel refinement module” [i.e., the computation of a loss for training of the GAN neural networks using the constraints implies a gradient is will be produced for optimizing the optical flow frame predictions]).
Mustikovela Shen and Chi are analogous art because they are both directed machine learning techniques used in spatial imaging (see, e.g., Mustikovela, paragraph [0074] and Chi [0026]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Chi to use a gradient that has a scene flow constraint term that increases in value with a temporal offset from an initial point in time. Doing so would have allowed Mustikovela to use Chi's method in order to “consider temporal consistency by applying adaptive processing to the optical flow maps used to generate the intermediate frames, and/or to the generated intermediate frames themselves, to focus processing on the high-error middle-most frames”, as suggested by Chi (see, e.g., Chi, paragraph [0027]).
Claims 9, 17 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Mustikovela and Shen, as applied to claims 1, 10 and 23 above, in view of Bailey (US 20210165938 A1; hereinafter Bailey).
Regarding claim 9, as discussed above, Mustikovela and Shen teaches the method of claim 1. However, Mustikovela and Shen fails to explicitly teach wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space.
Nevertheless, in the same field, analogous art Bailey teaches wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space (see, e.g., Bailey paragraph [0129] “For both the vanilla GAN and conditional GAN (CGAN) investigations, fully connected neural networks were implemented to model G and D. For the Vanilla GAN example, through hyper parameter optimization, the following architecture for generating wells was observed: Latent space: dimension = 128 distribution = normal” [i.e., 128-dimensional latent space is used] and paragraph [0180]: “Continuing with this example, the pre-trained GANs may be used further to generate 3D fluvial facies models constrained by facies interpretations in well locations… The size of each training model is 32 x 64 x 64 in z-, y-, and x- direction respectively” [i.e., the GANs use three-dimensional space for its output generation]).
Mustikovela Shen and Bailey are analogous art because they are both directed to machine learning techniques for image processing (see, e.g., Mustikovela, paragraph [0059] and Bailey, paragraph [0128]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Bailey to use the method for training machine learning models in an environment that includes multidimensional spaces with a three-dimensional space and a 128-dimensional space. Doing so would have allowed Mustikovela to use Bailey's method in order to “reproduce the complexity of many geological environments… improve the 'realism' of synthetic training images which may be another key benefit over conventional methods”, as suggested by Bailey (see, e.g., Bailey, paragraph [0093]).
Regarding claim 17, as discussed above, Mustikovela and Shen teaches the method of claim 10. However, Mustikovela and Shen fails to explicitly teach wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space.
Nevertheless, in the same field, analogous art Bailey teaches wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space (see, e.g., Bailey paragraph [0129] “For both the vanilla GAN and conditional GAN (CGAN) investigations, fully connected neural networks were implemented to model G and D. For the Vanilla GAN example, through hyper parameter optimization, the following architecture for generating wells was observed: Latent space: dimension = 128 distribution = normal” [i.e., 128-dimensional latent space is used] and paragraph [0180]: “Continuing with this example, the pre-trained GANs may be used further to generate 3D fluvial facies models constrained by facies interpretations in well locations… The size of each training model is 32 x 64 x 64 in z-, y-, and x- direction respectively” [i.e., the GANs use three-dimensional space for its output generation]).
Mustikovela Shen and Bailey are analogous art because they are both directed to machine learning techniques for image processing (see, e.g., Mustikovela, paragraph [0059] and Bailey, paragraph [0128]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Bailey to use the method for training machine learning models in an environment that includes multidimensional spaces with a three-dimensional space and a 128-dimensional space. Doing so would have allowed Mustikovela to use Bailey's method in order to “reproduce the complexity of many geological environments… improve the 'realism' of synthetic training images which may be another key benefit over conventional methods”, as suggested by Bailey (see, e.g., Bailey, paragraph [0093]).
Regarding claim 30, as discussed above, Mustikovela and Shen teaches the system of claim 23. However, Mustikovela and Shenfails to explicitly teach wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space.
Nevertheless, in the same field, analogous art Bailey teaches wherein the one or more multidimensional spaces comprises a three-dimensional space and a 128-dimensional space (see, e.g., Bailey paragraph [0129] “For both the vanilla GAN and conditional GAN (CGAN) investigations, fully connected neural networks were implemented to model G and D. For the Vanilla GAN example, through hyper parameter optimization, the following architecture for generating wells was observed: Latent space: dimension = 128 distribution = normal” and paragraph [0180]: “Continuing with this example, the pre-trained GANs may be used further to generate 3D fluvial facies models constrained by facies interpretations in well locations… The size of each training model is 32 x 64 x 64 in z-, y-, and x- direction respectively” [i.e., the GANs use three-dimensional space for its output generation]).
Mustikovela Shen and Bailey are analogous art because they are both directed to machine learning techniques for image processing (see, e.g., Mustikovela, paragraph [0059] and Bailey, paragraph [0128]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mustikovela to incorporate the teachings of Bailey to use the method for training machine learning models in an environment that includes multidimensional spaces with a three-dimensional space and a 128-dimensional space. Doing so would have allowed Mustikovela to use Bailey's method in order to “reproduce the complexity of many geological environments… improve the 'realism' of synthetic training images which may be another key benefit over conventional methods”, as suggested by Bailey (see, e.g., Bailey, paragraph [0093]).
Related prior arts:
Alwon et al. (US 20190302290 A1) teaches ¶ 91, an adversarial approach, two networks can try to optimize a different and opposing objective function, or loss function, in a zero-sum game (e.g., an actor-critic model). In such an example, as the discriminator changes its behavior, so does the generator, and vice versa, such that their losses push against each other.
Munoz et al. (US 20210125061 A1) teaches in ¶ 149, During the training of the GAN, the weights/parameters for the generator G may be updated along the gradient direction that minimizes a loss corresponding to D(G(z)). This model is referred to as the critic C.
ConclusionAny inquiry concerning this communication or earlier communications from the examiner should be directed to IMAD M KASSIM whose telephone number is (571)272-2958. The examiner can normally be reached 10:30AM-5:30PM, M-F (E.S.T.).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297 - 4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IMAD KASSIM/Primary Examiner, Art Unit 2129
1 As indicated in the Claim Interpretation section above, the term "a first critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
2 As indicated in the Claim Interpretation section above, the term "a second critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
3 As indicated in the Claim Interpretation section above, the term "a second critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
4 As indicated in the Claim Interpretation section above, the term "one or more critic models”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
5 As indicated in the Claim Interpretation section above, the term "a first critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
6 As indicated in the Claim Interpretation section above, the term "a second critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
7 As indicated in the Claim Interpretation section above, the term "one or more critic models”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
8 As indicated in the Claim Interpretation section above, the term "a first critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
9 As indicated in the Claim Interpretation section above, the term "the second critic model”, is interpreted as a means-plus function limitation lacking sufficient structural support in the claim. Accordingly, the term is interpreted as any software model, module or logic able to perform the claimed function.
Prosecution Insights