DETAILED ACTION
This Office Action is in response to the RCE filed on 10/20/2025.
Claims 1, 12, and 16 currently amended.
Claims 21 and 22 newly added.
Claims 1-9, 12-16, and 20-22 are currently pending in this application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
In reference to Applicant’s arguments on page(s) 13-17 regarding rejections made under 35 U.S.C. 101:
In the Final Office Action, claims 1-9, 12-16, and 20 were rejected under 35 U.S.C. § 101. The Applicant requests reconsideration and respectfully urges that the amended claims recite sufficient additional elements to integrate any abstract ideas into a practical application under at least Step 2A, Prong Two of the USPTO's Eligibility Guidelines.
In the recent USPTO Appeals Review Panel's decision in Ex parte Desjardins, USPTO Director Squires reversed the PTAB and provided new guidance for how AI- related patent claims should be evaluated under 35 U.S.C. § 101, and specifically Step 2A, Prong Two. The claims in Ex parte Desjardins were directed to a "method of training a machine learning model" including steps of "computing ... an approximation of a posterior distribution", "assigning ... a value to each of the plurality of parameters", "obtaining second training data..."and "training the machine learning model on the second machine learning task by training the machine leaning model on the second training data...." The Director Squires-led panel held that training steps that operate to "'effectively learn new tasks in succession whilst protecting knowledge about previous tasks"' should be considered to "constitute an improvement to how the machine learning model itself operates, and not, for example, the identified mathematical calculation." Ex parte Desjardins at pages 8-9. The Director Squires-led panel went on to criticize the PTAB's approach that considered training operations as abstract, stating "the panel essentially equated any machine learning with an unpatentable 'algorithm' and the remaining additional elements as 'generic computer component,' ... Examiners and panels should not evaluate claims at such a high level of generality." Ex parte Desjardins at page 9.
Similar to Ex parte Desjardins, the present claims provide an improvement to how a machine learning model is trained, which constitutes an improvement to how the machine learning model itself operates and is thereby a practical application under Step 2A, Prong Two.
As discussed in the specification, previous machine learning model training
techniques have suffered a number of shortcomings. First, previous techniques have relied upon training datasets including "labeled examples of misclassifications or incorrect properties" so the model can learn what a misclassification looks like. However, "it is difficult to obtain large numbers of labeled examples of misclassifications or incorrect properties." See specification pages 2, line 29 to page 3, line 10. Since the type of training datasets most useful for previous training techniques have generally not been available, previous techniques have produced poorly trained machine learning models that could not accurately reclassify elements that have been misclassified or are missing information.
Second, previous machine learning model training techniques have relied upon the existence of "labeled datasets (i.e. datasets where the correctness of each class, category or property is known and indicated)." See specification pages 2, lines 27-29. However, most datasets available to be used in training include elements associated with classes, categories or properties where the correctness of the association is unknown (i.e. there is no label indicating whether the association is correct or incorrect). The classes, categories or properties are usually mostly correct, but seldom always correct. This has caused previous training techniques to create models that simply learn to reproduce data errors and that are not able to accurately reclassify elements that have been misclassified or are missing information.
The Applicant addresses these problems, and thereby provides an improvement to how a machine learning model is trained. This is achieved by a non-conventional training loop that identifies outliers from determined prototypes and updates a class, category or property of the elements in the training infrastructure models for these outliers in each iteration of the training loop. A conventional training loop would not look for outliers from prototypes or perform update actions based on them. As held in Ex parte Desjardins, this sort of improvement to how a machine learning model is trained constitutes an improvement to how the machine learning model itself operates and is thereby a practical application under Step 2A, Prong Two.
Accordingly, the Applicant respectfully requests that the claims be found eligible under Step 2A, Prong Two.
As recited in MPEP 2106, elements "do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations." MPEP15 2106.04(a)(2)(III)(A) citing SRI Int'l, Inc. v. Cisco Systems, Inc., 930 F.3d 1295, 1304 (Fed. Cir. 2019).
Further, these limitations are not merely instructions to apply it or the idea of a solution or outcome. Specific operations are recited. For example, the outliers are corrected by "updating a class, category or property of the elements." Likewise, the non- linear feature-embedding mapping is refined by "repeating the training, determining prototypes, and identifying embeddings that are outliers and correcting the outliers, until a condition is met." These operations cannot be fairly considered a general call to apply it, or the general idea of a desired solution or outcome.
Accordingly, the Applicant respectfully urges that the above discussed limitations may not fairly be dismissed as just mental processes, instructions to apply it, or the idea of a solution or outcome, and are thereby available for substantive analysis under Step 2A, Prong Two.
Examiner’s response:
Applicant’s arguments have been fully considered and are found to be persuasive.
Applicant argues that the instant invention is similar to Ex parte Desjardins in that the instant application performs repeated iterations of training the model to learn new information while maintaining the information of previous iterations. Examiner agrees. The panel in Ex parte Desjardins held that training steps that operate to "'effectively learn new tasks in succession whilst protecting knowledge about previous tasks"' should be considered to "constitute an improvement to how the machine learning model itself operates, and not, for example, the identified mathematical calculation.", as noted above in Applicant’s arguments. Similar ideas can be seen in the instant application in that the model operation identifies outliers in the embeddings and corrects them by updating an element of the outliers and does this iterative process until a condition is satisfied .
Applicant argues that the improvement herein relies on the idea that datasets with intentionally labeled correlations of the data is hard to come by. This issue is remedied by the instant application in that any outliers are identified, corrected, and the model is retrained until a condition is satisfied. Examiner agrees that there is an improvement present within the instant application. The outlier correction as a core component of the training method of the model accounts for instances where a labeled dataset might not be entirely accurate with the labels, therefore causing the model to learn mistakes that are inherent to the dataset with no real way to identify those mistakenly labeled data points and with no real way to correct those data points that are assumed to be correct.
In light of the arguments presented, the rejections made under 35 U.S.C. 101 are withdrawn.
In reference to Applicant’s arguments on page(s) 17-21 regarding rejections made under 35 U.S.C. 103:
In the Office Action, claims 1-5, 7, 12, 13 and 20 were rejected under 35 U.S.C. §103(a) over Austern et al., U.S. Publication No. 2021/0073441 (hereinafter "Austern") in view of Sikka et al., U.S. Publication No. 2020/008224 (hereinafter "Sikka), in view of Agrafiotis et al, U.S. Publication No. 2002/0091655 (hereinafter "Agrafiotis") in further view of Pahde et al., "Multimodal Prototypical Networks for Few Shot Learning" (hereinafter "Pahde"), in still further view of Mopur et al., Publication No. 2022/0215289 (hereinafter "Mopur").
Austern, Sikka, Agrafiotis, Pahde and Mopur do not suggest at least the claimed "identifying embeddings learned by the neural network that are outliers from the determined prototypes and correcting the outliers by updating a class, category or property of the elements in the training infrastructure models corresponding to the outliers" and "refining, by the software, the non-linear feature-embedding mapping learned by the neural network based on the corrected outliers by repeating the training, determining prototypes, and identifying embeddings that are outliers and correcting the outliers, until a condition is met."
The "examiner must first set forth a prima facie case, supported by evidence, showing why the claims at issue would have been obvious in light of the prior art." See MPEP 2142 citing ACCO Brands Corp. v. Fellowes, Inc., 813 F.3d 1361, 1365-66, (Fed. Cir. 2016). All claim limitations must be considered. See MPEP 2143.03.
Here, the Examiner has not established a prima facie case of obviousness against at least the "identifying" and "refining" limitations. While the Applicant has expanded these limitations with additional details in the current Amendment, the previous version of the claims still recited "identifying embeddings learned by the neural network that are outliers from the determined prototypes and correcting the outliers by updating the elements corresponding to the outliers" and "refining, by the software, the non-linear feature-embedding mapping learned by the neural network ... based on the corrected outliers." However, no mention of this language is found in the Final Office Action dated Sept. 5, 2025. While the Final Office Action listed other limitations of claim 1 with citations to various prior art references, there is no listing of the "identifying" and "refining" limitations or citations. Accordingly, the Applicant respectfully urges that a prima facie case of obviousness has not been set forth.
The Applicant further urges that the "identifying" and "refining" limitations, in combination with all the other limitations of claim 1, represent a non-obvious advance over prior training techniques. As discussed above, previous training techniques have suffered a number of shortcomings. The Applicant addresses the problems of previous training techniques with a non-conventional training loop that identifies outliers from determined prototypes and updates a class, category or property of the elements in the training infrastructure models for these outliers in each iteration of the training loop. A conventional training loop would not look for outliers from prototypes or perform update actions based on them.
Accordingly, it is requested that the rejection of claim 1 under 35 U.S.C. §103(a) be reconsidered. Independent claim 12 should be distinguishable at least due to similar reasoning. The remaining claims are dependent claims. Such dependent claims should be allowable at least due to their dependency from allowable independent claims.
In the Final Office Action, claims 6, 8, 14 and 15 were rejected under 35 U.S.C. § 103(a) over Austern, Sikka, Agrafiotis, Pahde and Mopur in further view of Dabbura, "K Means Clustering: Algorithm, Applications, Evaluation Methods, and Drawbacks (hereinafter "Dabbura").
Claims 6, 8, 14 and 15 are dependent claims. Such dependent claims should be allowable at least due to their dependency from allowable independent claims.
In the Final Office Action, claims 9 and 16 were rejected under 35 U.S.C. §103(a) over Austern, Sikka, Agrafiotis, Pahde and Mopur in further view of Kruus et al., U.S. Publication No. 2021/0089924 (hereinafter "Kruus").
Independent claim 16 should be distinguishable from the references for reasons similar to claim 1. Kruus is only cited in the Final Office Action in relation to loss determination and repeating limitations. No mention is made of the identifying limitation. Accordingly, it is requested that the rejection of claim 16 under 35 U.S.C. §103(a) be reconsidered. The remaining claims are dependent claims. Such dependent claims should be allowable at least due to their dependency.
Examiner’s response:
Applicant’s arguments have been fully considered but are found to be not persuasive in light of the amendments made on the claims.
Applicant argues that the Examiner did not present a prima facie case of obviousness against the independent claims as the “identifying” and “refining” limitations were omitted from the previous office action. Examiner agrees. Examiner apologizes for accidentally omitting two whole limitations from the action, especially since they are marked as covered by the prior art in the Examiner’s notes. Rest assured that the limitations in question were fully examined and art was applied internally to the application, but they must have slipped through among the minutia of writing the action.
In light of the amendments made on the claims, the rejections made under 35 U.S.C. 103 are maintained and updated below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-5, 7, 12, 13, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Austern et al (US 20210073441 A1, hereinafter Austern), in view of Sikka et al (US 20200082224 A1, hereinafter Sikka), in view of Agrafiotis et al. (US 20020091655 A1, hereinafter Agrafiotis), in view of Pahde et al (Pahde, Frederik, et al. “Multimodal Prototypical Networks for Few-Shot Learning.” 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, 2021. Crossref, https://doi.org/10.1109/wacv48630.2021.00269., hereinafter Pahde) and in further view of Mopur et al (US 20220215289 A1, hereinafter Mopur).
Regarding Claim 1:
Austern teaches
A method for reclassifying an element of an infrastructure model that is misclassified or is missing a class, category or property to enable analytics, comprising: accessing one or more training infrastructure models that include a plurality of elements that model individual portions of infrastructure, wherein infrastructure is a physical structure or object to be built (Austern [0005]: "The methods may include accessing a floor plan demarcating a plurality of rooms. The methods may further include receiving a first functional requirement for at least one first room of the plurality of rooms and receiving a second functional requirement for at least one second room of the plurality of rooms");
using the one or more training infrastructure models, (Austern [0013]: "Disclosed embodiments, systems, methods, and computer readable media related to extracting data from a 2D floor plan and retaining it in a building information model are disclosed. The methods may include accessing a 2D floor plan demarcating a plurality of rooms.")
wherein the features are derived from metadata describing geometric, contextual or temporal aspects of the portion of infrastructure modeled by the elements (Austern [0121]: "BIM files may be used to generate and manage digital representations of physical and functional characteristics of places, buildings and objects. BIM objects may describe actual architectural elements such as walls, doors and furniture and equipment and can hold a various metadata, geometric and parametric information regarding them.")
correcting the outliers by updating a class, category or property of the elements in the training infrastructure models corresponding to the outliers (Austern [0351]: “semantic enrichment may include updating or augmenting a prior semantic designation. As described above, a floor plan may include one or more existing designations. When semantic enrichment is performed on the floor plan, one or more of these prior existing semantic designations may be updated by replacement with a new designation. Continuing the previous example, the prior designation of “washroom” may be updated to the new semantic designation of “bathroom.”; (EN): while Austern does not teach the correction of outliers specifically, Mopur cited below does deal with outliers and the updating of the elements of the model of Austern can reasonably be applied to the outliers of Mopur”)
for an infrastructure model that includes misclassified elements or elements that are missing classes, categories or properties, (Austern [0316]: “Embodiments may include processes for semantic enrichment of floor plans, which refers to a process of adding missing or misrepresented semantic information (e.g., classifications, labels, names, or other semantic designations)”)
determining, by the software, features for an element of the infrastructure model, (Austern [0131]: "For detecting architectural features in the floor plans and predicting features, various deep learning models may be used")
providing, by the software, an output that indicates that the element is misclassified or that the element is missing the class, category or property, (Austern [0316]: “Embodiments may include processes for semantic enrichment of floor plans, which refers to a process of adding missing or misrepresented semantic information (e.g., classifications, labels, names, or other semantic designations)”)
reclassifying, by the software, the element to the predicted class, category or property, (Austern [0316]: "Adding semantic designations to architectural plans may be a difficult and labor intensive task. Disclosed embodiments may simplify this process by semantically enriching floor plans to add new or change existing semantic designations associated with spaces of the floor plan. Embodiments include using a variety of automated approaches that consider geometric or other image properties of floor plans and use artificial intelligence models to semantically enrich floor plans.")
Austern does not distinctly disclose
a neural network to learn a non-linear feature- embedding mapping that maps features of elements of the training infrastructure models to embeddings
and the embeddings are multi-dimensional numerical representations distributed in multi- dimensional embedding space such that distance between the embeddings is meaningful to a class, category or property of the elements of the infrastructure models
applying, by the software, the non-linear feature-embedding mapping to the determined features to produce an embedding for the element,
However, Agrafiotis teaches
a neural network to learn a non-linear feature- embedding mapping that maps features of elements of the training infrastructure models to embeddings (Agrafiotis [0090]: "For a nonlinear projection from n to m dimensions, a simple 3-layer network with n input and m output units can be employed. The network is trained to reproduce the input/output coordinates produced by the iterative algorithm, and thus encodes the mapping in its synaptic parameters in a compact, analytical manner.")
and the embeddings are multi-dimensional numerical representations distributed in multi- dimensional embedding space such that distance between the embeddings is meaningful to a class, category or property of the elements of the infrastructure models (Agrafiotis [0097]: "In step 320, a set of reference points P={c.sub.1, i=1,2, . . . c; c.sub.1 .di-elect cons. R.sup.n} is determined. In an embodiment of the invention, the reference points c.sub.i are determined using a clustering algorithm")
applying, by the software, the non-linear feature-embedding mapping to the determined features to produce an embedding for the element, (Agrafiotis [0090]: "For a nonlinear projection from n to m dimensions, a simple 3-layer network with n input and m output units can be employed. The network is trained to reproduce the input/output coordinates produced by the iterative algorithm, and thus encodes the mapping in its synaptic parameters in a compact, analytical manner."; [0097]: "In step 320, a set of reference points P={c.sub.1, i=1,2, . . . c; c.sub.1 .di-elect cons. R.sup.n} is determined. In an embodiment of the invention, the reference points c.sub.i are determined using a clustering algorithm")
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the nonlinear mapping of Agrafiotis in order to properly reduce the dimensions of the input(s) while maintaining the relationship of the data (Agrafiotis [0025]: “A method and computer product is presented for mapping input patterns of high dimensionality into a lower dimensional space so as to preserve the relationships between these patterns in the higher dimensional space”), as well as to provide a distance based classification calculation that is meaningful to the prediction.
Austern + Agrafiotis does not distinctly disclose
training using weakly supervised learning, by software executing on one or more computing devices
However, Sikka teaches
training using weakly supervised learning, by software executing on one or more computing devices (Sikka [0044]: "The method 550 begins at 552 and proceeds to 554 where at least one weakly labeled image is received. The method proceeds to 506 where the at least one weakly labeled image is analyzed using a deep network model and weakly supervised learning to determine a feature map of the at least one weakly labeled image.")
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern + Agrafiotis with the weakly supervised learning and weakly labeled image of Sikka. Sikka is beneficial for Austern + Agrafiotis because it provides a method for weakly supervised learning to improve the performance of the model (Sikka [0037]: “In particular we observe a consistent improvement across all data splits by using WSL, the hike in performance by using WSL is 4% absolute points”).
Austern + Agrafiotis + Sikka does not distinctly disclose
determining, by the software, prototypes for a plurality of groups of elements associated with different classes, categories or properties based on the embeddings learned by the neural network from weakly supervised learning, wherein a prototype is a single representation in the multi-dimensional embedding space that embeddings of a group cluster around;
refining, by the software, the non-linear feature-embedding mapping learned by the neural network based on the corrected outliers by repeating the training, determining prototypes, and identifying embeddings that are outliers and correcting the outliers, until a condition is met;
predicting, by the software, a class, category or property of the element based on distance in multi-dimensional embedding space between the embedding for the element and the prototype of each of the plurality of groups,
However, Pahde teaches
determining, by the software, prototypes for a plurality of groups of elements associated with different classes, categories or properties based on the embeddings learned by the neural network from weakly supervised learning, wherein a prototype is a single representation in the multi-dimensional embedding space that embeddings of a group cluster around (Pahde [Section 4.2: Classification; Figure 1]: "We predict the class membership of test samples by calculating the nearest prototype in the embedding space ϕ (see Eq. 2). As distance function we use cosine distance.”; Figure 1 of Pahde shows an embedding space with 3 prototypes and a distance for new elements being added; (EN): it is noted that the “prototypes” of applicant's disclosure are analogous to centroids used in clustering algorithms);
refining, by the software, the non-linear feature-embedding mapping learned by the neural network based on the corrected outliers by repeating the training, determining prototypes, and identifying embeddings that are outliers and correcting the outliers, until a condition is met (Pahde [Section 4.2: Classification]: “We predict the class membership of test samples by calculating the nearest prototype in the embedding space ϕ (see Eq. 2). As distance function we use cosine distance. To average visual and textual prototypes we set λ = 1 (see Eq. 10) and repeat this step 10 times, updating Gt in every iteration. Hence, in each iteration we reuse real samples from S-k-train, combined with novel generated samples given an updated generator Gt.”; (EN): while Pahde does not teach the identification of outliers, this same repeated process can be applied to the outliers that are identified in Mopur, cited below);
predicting, by the software, a class, category or property of the element based on distance in multi-dimensional embedding space between the embedding for the element and the prototype of each of the plurality of groups, (Pahde [Section 4.2: Classification; Figure 1]: "We predict the class membership of test samples by calculating the nearest prototype in the embedding space ϕ (see Eq. 2). As distance function we use cosine distance.”; Figure 1 of Pahde shows an embedding space with 3 prototypes and a distance for new elements being added; (EN): it is noted that the “prototypes” of applicant's disclosure are analogous to centroids used in clustering algorithms);
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern + Agrafiotis + Sikka with distance based classification of Pahde. The addition of Pahde is beneficial for Austern + Agrafiotis + Sikka because Pahde provides a distance based method in order to properly predict which prototype a given element belongs to (Pahde [Fig. 1]: “Cross-modal generated representations can condense the embedding space and move the visual prototypes pI towards more reliable multimodal prototypes pM and it improves the classification accuracy on unseen test samples”).
Austern + Agrafiotis + Sikka + Pahde does not distinctly disclose
identifying embeddings learned by the neural network that are outliers from the determined prototypes
comparing, by the software, the predicted class, category or property of the element based on the distance between the embedding for the element and the prototype to a currently associated, category or property of the element
However, Mopur teaches
identifying embeddings learned by the neural network that are outliers from the determined prototypes (Mopur [0034]: "The outlier detection unit 214 determines outliers from the clusters, based on one or more factors including maximum distance from one or more densely populated clusters, count of values of the data points, and comparison of the values with watermarks predefined in baseline data.")
comparing, by the software, the predicted class, category or property of the element based on the distance between the embedding for the element and the prototype to a currently associated, category or property of the element (Mopur [0034]: "The outlier detection unit 214 determines outliers from the clusters, based on one or more factors including maximum distance from one or more densely populated clusters, count of values of the data points, and comparison of the values with watermarks predefined in baseline data.")
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern + Agrafiotis + Sikka + Pahde with the system and method of handling outliers for machine learning models of Mopur. The addition of Mopur is beneficial for Austern + Agrafiotis + Sikka + Pahde because Mopur provides a method for detecting outliers in data and handling them so they no longer affect prediction behavior of the ML model (Mopur [0004]: “Outliers present in input data streams and data stream drifts could affect prediction behaviour of ML models which are pre-trained using labelled data.”).
Regarding Claim 2:
Austern teaches
The method of claim 1, wherein the output includes an
indication that the element is misclassified, and a suggestion of reclassifications to the predicted class, category or property (Austern [0316]: “Embodiments may include processes for semantic enrichment of floor plans, which refers to a process of adding missing or misrepresented semantic information (e.g., classifications, labels, names, or other semantic designations)”).
Regarding Claim 3:
Austern teaches
The method of claim 1, wherein the output includes an indication the element is missing a class, category or property, and a suggestion of classification to the predicted class, category or property (Austern [0316]: “Embodiments may include processes for semantic enrichment of floor plans, which refers to a process of adding missing or misrepresented semantic information (e.g., classifications, labels, names, or other semantic designations)”).
Regarding Claim 4:
Austern does not distinctly disclose
The method of claim 1, wherein the output includes an embedding-based visualization in which the embedding of the element and embeddings of other elements of the plurality of groups are plotted as points in multi-dimensional space.
However, Mopur teaches
The method of claim 1, wherein the output includes an embedding-based visualization in which the embedding of the element and embeddings of other elements of the plurality of groups are plotted as points in multi-dimensional space (Mopur [0008]: “FIG. 3A illustrates clusters prepared from reconstruction errors corresponding to a first batch of images, in accordance with an embodiment of the present disclosure”; [0009]: “FIG. 3B illustrates a sample representation of clusters prepared for the first batch of images through affinity propagation, using an optimal preference value, in accordance with an embodiment of the present disclosure”; [0010]: “FIG. 4A illustrates clusters prepared from reconstruction errors corresponding to a second hatch of images, in accordance with an embodiment of the present disclosure”; [0011]: “FIG. 4B illustrates a sample representation of clusters prepared for the second batch of images through affinity propagation, using an optimal preference value, in accordance with an embodiment of the present disclosure”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the system and method of handling outliers for machine learning models of Mopur. The addition of Mopur is beneficial for Austern because Mopur provides a method for detecting outliers in data and handling them so they no longer affect prediction behavior of the ML model (Mopur [0004]: “Outliers present in input data streams and data stream drifts could affect prediction behaviour of ML models which are pre-trained using labelled data.”).
Regarding Claim 5:
Austern does not distinctly disclose
The method of claim 4, wherein the providing further comprises: reducing dimensionality of the embeddings to produce a set of points that represent elements, selecting a mapping of visual features of points to aspects of the elements
However, Agrafiotis teaches
The method of claim 4, wherein the providing further comprises: reducing dimensionality of the embeddings to produce a set of points that represent elements, selecting a mapping of visual features of points to aspects of the elements; (Agrafiotis [0021]: “Thus, for a nonlinear projection from n to m dimensions, a standard 3-layer neural network with n input and m output units is used. Each n-dimensional object is presented to the input layer, and its coordinates on the m-dimensional nonlinear map are obtained by the respective units in the output layer”; (EN): it is noted that m < n in this context).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the nonlinear mapping of Agrafiotis in order to properly reduce the dimensions of the input(s) while maintaining the relationship of the data (Agrafiotis [0025]: “A method and computer product is presented for mapping input patterns of high dimensionality into a lower dimensional space so as to preserve the relationships between these patterns in the higher dimensional space”), as well as to provide a distance based classification calculation that is meaningful to the prediction.
Austern and Agrafiotis do not distinctly disclose
And plotting, by the software, the set of points with the selected visual features to produce the embedding-based visualization.
However, Mopur teaches
And plotting, by the software, the set of points with the selected visual features to produce the embedding-based visualization (Mopur [0008]: “FIG. 3A illustrates clusters prepared from reconstruction errors corresponding to a first batch of images, in accordance with an embodiment of the present disclosure”; [0009]: “FIG. 3B illustrates a sample representation of clusters prepared for the first batch of images through affinity propagation, using an optimal preference value, in accordance with an embodiment of the present disclosure”; [0010]: “FIG. 4A illustrates clusters prepared from reconstruction errors corresponding to a second hatch of images, in accordance with an embodiment of the present disclosure”; [0011]: “FIG. 4B illustrates a sample representation of clusters prepared for the second batch of images through affinity propagation, using an optimal preference value, in accordance with an embodiment of the present disclosure”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the system and method of handling outliers for machine learning models of Mopur. The addition of Mopur is beneficial for Austern because Mopur provides a method for detecting outliers in data and handling them so they no longer affect prediction behavior of the ML model (Mopur [0004]: “Outliers present in input data streams and data stream drifts could affect prediction behaviour of ML models which are pre-trained using labelled data.”).
Regarding Claim 7:
Austern does not distinctly disclose
The method of claim 1, wherein the determined features are represented in k- dimensional space and the multi-dimensional embedding space is n-dimensional space, with n different from k
However, Agrafiotis teaches
The method of claim 1, wherein the determined features are represented in k- dimensional space and the multi-dimensional embedding space is n-dimensional space, with n different from k (Agrafiotis [0090]: “For a nonlinear projection from n to m dimensions, a simple 3-layer network with n input and m output units can be employed. The network is trained to reproduce the input/output coordinates produced by the iterative algorithm, and thus encodes the mapping in its synaptic parameters in a compact, analytical manner; (EN): examiner is interpreting the different dimensional spaces to mean the following: k space is the original dimension space (unreduced), n dimensional space is the dimensional space of the reduced embedding(s), x dimensional space is the visualization space).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the nonlinear mapping of Agrafiotis in order to properly reduce the dimensions of the input(s) while maintaining the relationship of the data (Agrafiotis [0025]: “A method and computer product is presented for mapping input patterns of high dimensionality into a lower dimensional space so as to preserve the relationships between these patterns in the higher dimensional space”), as well as to provide a distance based classification calculation that is meaningful to the prediction.
Regarding Claim 12:
Due to claim language similar to that of Claim 1, Claim 12 is rejected for the same reasons as presented above in the rejection of Claim 1.
Regarding Claim 20:
Due to claim language similar to that of Claim 4, Claim 20 is rejected for the reasons stated above in the rejection of Claim 4.
Claim Rejections - 35 USC § 103
Claim(s) 6, 8, and 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Austern, Agrafiotis, Sikka, Pahde, and Mopur as applied to claims 1 and 12 above, and further in view of Daburra (Dabbura, Imad. “K MeansClustering:Algorithm,Applications,EvaluationMethods,andDrawbacks.”Medium,September17,2018. https://towardsdatascience.com/k-means-clustering-algorithm-applications-evaluation-methods-and-drawbacks-aa03e644b48a.).
Regarding Claim 6:
Austern does not distinctly disclose
The method of claim 4, wherein the visual features include colors, and the embedding- based visualization maps class, category or property to colors, such that points having a common class, category or property share a common color.
However, Dabbura teaches
The method of claim 4, wherein the visual features include colors, and the embedding- based visualization maps class, category or property to colors, such that points having a common class, category or property share a common color (Dabbura: Fig. 4, the grid of graphs, shows the 2D visualization of data being clustered around 2 centroids (or prototypes) of the data).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the output display of Dabbura in order to properly show the classification of the data in an easy to interpret manner (Dabbura: Fig. 4, the grid of graphs, showing the data being clustered around 2 centroids).
Regarding Claim 8:
Austern does not distinctly disclose
The method of claim 7, wherein the output includes an embedding-based visualization in which the embedding of the element and embeddings of other elements of the plurality of groups are plotted as points in x-dimensional space, and x<n.
However, Dabbura teaches
The method of claim 7, wherein the output includes an embedding-based visualization in which the embedding of the element and embeddings of other elements of the plurality of groups are plotted as points in x-dimensional space, and x<n (Dabbura: Figure(s) entitled “Silhouette analysis using k = “ show the plots of data in a 2 dimensional space while the number of clusters (n) varies from 2 to 4).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the output display of Dabbura in order to properly show the classification of the data in an easy to interpret manner (Dabbura: Figure(s) entitled “Silhouette analysis using k = “ showing how the data changes classification with the introduction of new centroids).
Regarding Claim 13:
Due to claim language similar to that of Claim 6, Claim 13 is rejected for the same reason(s) as presented above in the rejection of Claim 6.
Regarding Claim 14:
Austern does not distinctly disclose
The method of claim 13, wherein embedding-based visualization identifies misclassifications as points more than a given distance from other points having a same color.
However, Dabbura teaches
The method of claim 13, wherein embedding-based visualization identifies misclassifications as points more than a given distance from other points having a same color (Dabbura: Fig. 4, the grid of graphs, shows that over multiple iterations of clustering, the points around the origin of the graph(s) (coordinates (0.0, 0.0)), continue to switch between the blue cluster and green cluster as more information is provided to the network, meaning that the points that switched clusters are misclassified).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the output display of Dabbura in order to properly show the classification of the data in an easy to interpret manner (Dabbura: Fig. 4, the grid of graphs, showing the data being clustered around 2 centroids).
Regarding Claim 15:
Due to claim language similar to that of Claims 7 and 8, Claim 15 is rejected for the same reasons as presented above in the rejections of Claims 7 and 8.
Claim Rejections - 35 USC § 103
Claim(s) 9, 16, 21, and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Austern, Agrafiotis, Sikka, Pahde, and Mopur as applied to claims 1 and 12 above, and further in view of Kruus et al (US 20210089924 A1, hereinafter Kruus).
Regarding Claim 9:
Austern teaches
The method of claim 1, wherein the training using weakly supervised learning further comprises: sampling, by the software, training elements from one or more groups of a training infrastructure model to produce a training set of features for the training elements; (Austern [0129]: “Training data may take many forms including, for example, the annotation of elements including but not limited to various architectural features (e.g. doors, door sills, windows, walls, rooms, etc.)”; Austern [0131]: “Annotated floor plans may be divided into training and validation sets.”; (EN): the training process includes selecting an amount of data from one of the sets of training data).
Austern does not distinctly disclose
providing, by the software, the training set of features for the training elements as input to the neural network, for each of the groups, computing a prototype
and calculating a probability of an embedding belonging to each group based on a distance in multi-dimensional space between the embedding and the prototypes for each of the groups, and determining a predicted group from the probability of belonging to each group
However, Agrafiotis teaches
providing, by the software, the training set of features for the training elements as input to the neural network, for each of the groups, computing a prototype (Agrafiotis [0096]: “The training phase begins at step 305. In step 310, a random set of points is extracted from the set of input patterns. In step 315, the points X.sub.i are mapped from R.sup.n to R.sup.m using the iterative nonlinear mapping algorithm described in Section II. This mapping serves to define a training set T of ordered pairs”, Agrafiotis [0097]: “In step 320, a set of reference points is determined. In an embodiment of the invention, the reference points c.sub.i are determined using a clustering algorithm”).
and calculating a probability of an embedding belonging to each group based on a distance in multi-dimensional space between the embedding and the prototypes for each of the groups, and determining a predicted group from the probability of belonging to each group (Agrafiotis [0077]: “This deterministic criterion may be replaced by a stochastic or probabilistic one in which the target distance is selected either randomly or with a probability that depends on the difference between the current distance and the two nearest range boundaries”)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the structural design system of Austern with the nonlinear mapping of Agrafiotis in order to properly reduce the dimensions of the input(s) while maintaining the relationship of the data (Agrafiotis [0025]: “A method and computer product is presented for mapping input patterns of high dimensionality into a lower dimensional space so as to preserve the relationships between these patterns in the higher dimensional space”), as well as to provide a distance based classification calculation that is meaningful to the prediction.
Austern + Agrafiotis does not distinctly disclose
and determining a loss based on a comparison of the predicted group and a true group, the loss used to provide feedback to the neural network;
And repeating the sampling, providing, computing, calculating and determining over the course of a plurality of training steps to train the neural network
However, Kruus teaches
and determining a loss based on a comparison of the predicted group and a true group, the loss used to provide feedback to the neural network; (Kruus [0201]: “the loss function for classification is chosen as negative log likelihood loss”; (EN): it is noted that applicant also chose a negative log likelihood loss function in their disclosure).
And repeating the sampling, providing, computing, calculating and determining over the course of a plurality of training steps to train the neural network (Kruus [0132]: “Considering the large sample size(60000), here we use first 100 samples for the current small experiment”; (EN): using the first x number of samples out of a given sample size reads on “repeating sampling”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the nonlinear mapping of Agrafiotis and Austern with the sets of loss function of Kruus in order to improve the robustness of the network and maximize classification likelihood (Kruus [0143]: “The classification loss goes down to smaller than 0.1 (comparable with standard CNN) in this case”).
Regarding Claim 16:
Due to claim language similar to that of claims 1 and 12, Claim 16 is rejected for the same reasons as presented above in the rejections of claims 1 and 12, with the exception of the limitations covered below.
Austern + Agrafiotis + Sikka + Pahde + Mopur does not distinctly disclose
determining, by the software, a loss based on a comparison of the predicted group and a true group, the loss used to provide feedback to the neural network;
repeating the sampling, providing, determining the first subset, determining the predicted group and determining the loss over the course of a plurality of training steps to train the neural network using weakly supervised learning and, using the trained neural network is usable to classify elements or predict properties of elements
However, Kruus teaches
determining, by the software, a loss based on a comparison of the predicted group and a true group, the loss used to provide feedback to the neural network (Kruus [0201]: “the loss function for classification is chosen as negative log likelihood loss”; (EN): it is noted that applicant also chose a negative log likelihood loss function in their disclosure);
repeating the sampling, providing, determining the first subset, determining the predicted group and determining the loss over the course of a plurality of training steps to train the neural network using weakly supervised learning and, using the trained neural network is usable to classify elements or predict properties of elements (Kruus [0132]: “Considering the large sample size(60000), here we use first 100 samples for the current small experiment”; (EN): using the first x number of samples out of a given sample size reads on “repeating sampling”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the nonlinear mapping of Agrafiotis and Austern with the sets of loss function of Kruus in order to improve the robustness of the network and maximize classification likelihood (Kruus [0143]: “The classification loss goes down to smaller than 0.1 (comparable with standard CNN) in this case”).
Regarding Claim 21:
Due to claim language similar to that of Claims 1, 12, and 16, Claim 21 is rejected for the same reasons as presented above in the rejections of Claims 1, 12, and 16.
Regarding Claim 22:
Due to claim language similar to that of Claim 9, Claim 22 is rejected for the same reasons as presented above in the rejection of Claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 11151386 B1 – systems and methods for automated identification and tagging of video content
US 20210295091 A1 – system and method are directed to a prototypical contrastive learning (PCL)
WO 2018093935 A1 – Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network
US 8744125 B2 – Clustering-based Object Classification
US 20030147558 A1 – A method for classification of image regions by probabilistic merging of a class probability map and a cluster probability map
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COREY M SACKALOSKY whose telephone number is (703)756-1590. The examiner can normally be reached M-F 7:30am-3:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COREY M SACKALOSKY/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128