DETAILED ACTION
This office action is responsive to the above identified application filed 9/29/2023. The application contains claims 1-20, all examined and rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Claim 1 is rejected under 35 USC 101 because the claimed inventions are directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
While independent claims 1, 10 and 16 are each directed to a statutory category, they recites a series of steps which appears to be directed to an abstract idea (mental process, mathematical concept).
Claims 1-20 are rejected under 35 U.S.C. § 101 because the instant application is directed to non-patentable subject matter. Specifically, the claims are directed toward at least one judicial exception without reciting additional elements that amount to significantly more than the judicial exception. The rationale for this determination is in accordance with the guidelines of USPTO, applies to all statutory categories, and is explained in detail below.
When considering subject matter eligibility under 35 U.S.C. 101, (1) it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. If the claim does fall within one of the statutory categories, (2a) it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), and if so (2b), it must additionally be determined whether the claim is a patent-eligible application of the exception. If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim amounts to significantly more than the abstract idea itself. Examples of abstract ideas include certain methods of organizing human activities; a mental processes; and mathematical concepts, (2019 PEG)
STEP 1.
Per Step 1, the claims are determined to include process and machine as in independent Claims 1, 10, and 16, and in the therefrom dependent claims. Therefore, the claims are directed to a statutory eligibility category.
At step 2A, prong 1, The invention is directed to steps which is akin to Mental Process and mathematical concepts (see Alice), As such, the claims include an abstract idea. When considering the limitations individually and as a whole the limitations directed to the abstract idea are:
Claim 1
“learning, first visualization latent-space features of different datasets represented in a first two-dimensional latent space and second visualization latent-space features of different computation pipelines represented in a second two- dimensional latent space” (Mental process);
“modeling, dataset-pipeline interactions between the different datasets and the different computation pipelines based on the first visualization latent-space features and the second visualization latent-space features” (Mental process);
“learning, relationships between the first visualization latent-space features and the second visualization latent-space features based on modeling the dataset-pipeline interactions” (Mental process, observation, evaluation and judgment).
Claim 16
learning, relationships between first visualization latent-space features of different datasets represented in a first two-dimensional latent space and second visualization latent-space features of different computation pipelines represented in a second two-dimensional latent space (Mental process);
predicting, performance data of the different computation pipelines with respect to the different datasets based on the relationships(Mental process, observation, evaluation and judgment)
generating, a visual representation of the relationships and the performance data, the visual representation comprising: latitude and longitude data that are indicative of the relationships; and altitude data that are indicative of the performance data (mental process, human are capable of provide visual representation of data using a pen and paper).
The claim recites additional elements as
Claim 1
“by at least one computing device” (“Using a computer as a tool to perform a mental process”, MPEP 2106.04(a)(2)(III)(C)).
Claim 16
“by at least one computing device” (“Using a computer as a tool to perform a mental process”, MPEP 2106.04(a)(2)(III)(C)).
This judicial exception is not integrated into a practical application. The elements are recited at a high level of generality, i.e. a generic computing system performing generic functions including generic processing of data. Accordingly the additional elements do not integrate the abstract into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore the claims are directed to an abstract idea. (2019 Revised Patent Subject Matter Eligibility Guidance ("2019 PEG"). Thus, under Step 2A of the Mayo framework, the Examiner holds that the claims are directed to concepts identified as abstract.
STEP 2B.
Because the claims include one or more abstract ideas, the examiner now proceeds to Step 2B of the analysis, in which the examiner considers if the claims include individually or as an ordered combination limitations that are "significantly more" than the abstract idea itself. This includes analysis as to whether there is an improvement to either the "computer itself," "another technology," the "technical field," or significantly more than what is "well-understood, routine, or conventional" (WURC) in the related arts.
The instant application includes in Claim 1 additional steps to those deemed to be abstract idea(s).
When taken the steps individually, these steps are:
“by at least one computing device” (“Using a computer as a tool to perform a mental process” (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2));
In the instant case, Claim 1 is directed to above mentioned abstract idea. Technical functions such as receiving, and extracting are common and basic functions in computer technology. The individual limitations are recited at a high level and do not provide any specific technology or techniques to perform the functions claimed.
In addition, when the claims are taken as a whole, as an ordered combination, the combination of steps does not add "significantly more" by virtue of considering the steps as a whole, as an ordered combination. The instant application, therefore, still appears only to implement the abstract idea to the particular technological environments using what is well-understood, routine, and conventional in the related arts. The steps are still a combination made to the abstract idea. The additional steps only add to those abstract ideas using well understood and conventional functions, and the claims do not show improved ways of, for example, an unconventional non-routine functions for analyzing model operations or updating the model that could then be pointed to as being "significantly more" than the abstract ideas themselves.
Moreover, Examiner was not able to identify any "unconventional" steps, which, when considered in the ordered combination with the other steps, could have transformed the nature of the abstract idea previously identified. The instant application, therefore, still appears to only implement the abstract ideas to the particular technological environments using what is well-understood, routine, and conventional (WURC) in the related arts.
Further, note that the limitations, in the instant claims, are done by the generically
recited computing devices. The limitations are merely instructions to implement the abstract idea on a computing device that is recited in an abstract level and require no more than a generic computing devices to perform generic functions.
Claim 10 recites a “computing device, comprising: a memory device to store computer-readable instructions thereon; and at least one processing device” configured to perform the same method as set forth in claim 1, the added element of “computing device, comprising: a memory device to store computer-readable instructions thereon; and at least one processing device” do not transform the judicial exception into a practical application because they are tantamount to a mere instruction to apply the judicial exception to a generic computer. The additional elements are also not sufficient to amount to significantly more than the judicial exception because the action of implementing the method on a general purpose computer with at least one processor and at least one memory is tantamount to a mere instruction to apply the judicial exception to a computer.
Claim 10 is therefore rejected according to the same findings and rationale as provided above.
CONCLUSION
It is therefore determined that the instant application not only represents an abstract idea identified as such based on criteria defined by the Courts and on USPTO examination guidelines, but also lacks the capability to bring about "Improvements to another technology or technical field" (Alice), bring about "Improvements to the functioning of the computer itself" (Alice), "Apply the judicial exception with, or by use of, a particular machine" (Bilski), "Effect a transformation or reduction of a particular article to a different state or thing" (Diehr), "Add a specific limitation other than what is well-understood, routine and conventional in the field" (Mayo), "Add unconventional steps that confine the claim to a particular useful application" (Mayo), or contain "Other meaningful limitations beyond generally linking the use of the judicial exception to a particular technological environment" (Alice), transformed a traditionally subjective process performed by humans into a mathematically automated process executed on computers (McRO), or limitations directed to improvements in computer related technology, including claims directed to software (Enfish).
The dependent claims, when considered individually and as a whole, likewise do not provide "significantly more" than the abstract idea for similar reasons as the independent claim.
claims 2 disclose “learning the first visualization latent-space features and the second visualization latent-space features comprises: implementing, by the at least one computing device, two structurally symmetric variational autoencoders in parallel to respectively learn the first visualization latent-space features in the first two-dimensional latent space and the second visualization latent-space features in the second two-dimensional latent space.” (training a system which is a high-generic computer software process of training data. This limitation does not amount to significantly more than the judicial exception, see MPEP 2106.05 (f)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 3 disclose “modeling the dataset-pipeline interactions comprises: implementing, by the at least one computing device, a neural collaborative filtering network to predict performance metrics of the different computation pipelines with respect to the different datasets based on the first visualization latent-space features and the second visualization latent-space features” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h) and mere instructions to “apply” the abstract ideas, which cannot provide an inventive concept. See MPEP 2106.05(f)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 4 disclose “modeling the dataset-pipeline interactions comprises: implementing, by the at least one computing device, a neural collaborative filtering network to model the dataset-pipeline interactions using a neural collaborative filtering process comprising generalized matrix factorization of a recommendation matrix generated by the neural collaborative filtering network and multi-layer projection of the first visualization latent-space features and the second visualization latent-space features” (merely indicates a field of use or technological environment in which the judicial exception is performed and fails to add an inventive concept to the claims. See MPEP 2106.05(h) and mere instructions to “apply” the abstract ideas, which cannot provide an inventive concept. See MPEP 2106.05(f)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 5 disclose “generating, by the at least one computing device, a visual representation of the relationships and the dataset-pipeline interactions, the visual representation comprising: latitude and longitude data that are indicative of the relationships; and altitude data that are indicative of the dataset-pipeline interactions” (Mental process as human is capable of generating visual presentation using a pen and paper and the “by the at least one computing device” (“Using a computer as a tool to perform a mental process” (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 6 disclose “generating, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), meta-data vectors respectively corresponding to the different datasets, the meta-data vectors being generated based on respective summary statistics data of the different datasets” (Mental process, mathematical concept); and generating, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), embedding vectors respectively corresponding to the different computation pipelines based on data indicative of different pipeline component candidates of the different computation pipelines (Mental process, mathematical concept); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 7 disclose “generating, by the at least one computing device using a first of two structurally symmetric variational autoencoders (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), first visualization latent-space representations of the different datasets in the first two-dimensional latent space based on meta-data vectors respectively corresponding to the different datasets (Mental process); and generating, by the at least one computing device using a second of the two structurally symmetric variational autoencoders (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), second visualization latent-space representations of the different computation pipelines in the second two-dimensional latent space based on embedding vectors respectively corresponding to the different computation pipelines (Mental process), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 8 disclose “learning, by the at least one computing device using a first of two structurally symmetric variational autoencoders (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), the first visualization latent-space features in the first two- dimensional latent space based on first visualization latent-space representations that correspond to and represent the different datasets in the first two-dimensional latent space (Mental process); and learning, by the at least one computing device using a second of the two structurally symmetric variational autoencoders (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), the second visualization latent-space features in the second two-dimensional latent space based on second visualization latent-space representations that correspond to and represent the different computation pipelines in the second two-dimensional latent space (Mental process); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 9 disclose “training, by the at least one computing device, a joint variational autoencoder neural collaborative filtering network to learn the relationships using a hybrid loss function that drives the joint variational autoencoder neural collaborative filtering network toward achieving at least one of: a defined cluster-wised ranking prediction accuracy; a defined visualization latent-space representation generation accuracy in the first two- dimensional latent space and the second two-dimensional latent space; or a defined visualization profile continuity and smoothness of a visual representation of the relationships and the dataset-pipeline interactions” (training a system which is a high-generic computer software process of training data. This limitation does not amount to significantly more than the judicial exception, see MPEP 2106.05 (f)), It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 18 disclose “determining, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), a three-dimensional predicted visualization profile for each meshgrid cell in the two-dimensional meshgrid (Mental process); and generating, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), a colormap for each meshgrid cell of the two-dimensional meshgrid based on the three-dimensional predicted visualization profile (Mental process), the colormap being indicative of predicted performance data of one of the different computation pipelines with respect to one of the different datasets (data description, which is directed to generally linking the use of a judicial exception to a particular technological environment or type or source of data or field of use MPEP 2106.05(h)); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 19 disclose “arranging, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), meshgrid cells in the two-dimensional meshgrid based at least one of the relationships or the performance data, wherein subsets of the meshgrid cells having at least one of similar relationships or similar performance data are arranged adjacent to one another (Mental process); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea, claims 20 disclose “projecting, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), the two-dimensional meshgrid to a three-dimensional space (Mental process); and generating, by the at least one computing device (“Using a computer as a tool to perform a mental process”, MPEP 2106.05(f)(2)), a sphere-shaped visual representation of the relationships and the performance data based on projecting the two-dimensional meshgrid to the three-dimensional space (Mental process); It does not integrate the abstract idea into a practical application and did not add significantly more to the abstract idea. Claims 11-15 is similar is scope to claims 2-5 and 9; therefore they rejected under similar rationale.
The dependent claims which impose additional limitations also fail to claim patent eligible subject matter because the limitations cannot be considered statutory. The dependent claim(s) have been examined individually and in combination with the preceding claims, however they do not cure the deficiencies of claim 1; where all claims are directed to the same abstract idea, "addressing each claim of the asserted patents [is] unnecessary." Content Extraction &. Transmission LLC v, Wells Fargo Bank, Natl Ass'n, 776 F.3d 1343, 1348 (Fed. Cir. 2014). If applicant believes the dependent claims are directed towards patent eligible subject matter, they are invited to point out the specific limitations in the claim that are directed towards patent eligible subject matter. Claims for the other statutory classes are similarly analyzed.
For at least these reasons, the claimed inventions of each of dependent claims 2-9, 11-15, and 17-20,are directed or indirect to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more and are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2].
With regard to Claim 1,
D1 teach a method to analyze computation pipelines and datasets (Abstract), the method comprising:
learning, by at least one computing device (P. 6, 4.1, “we limited the maximum training time of each individual model within a pipeline to 30 seconds and its memory consumption to 16GB. Because of network failures and the cluster occasionally running out of memory”, P. 7, 4.2, “The latent space was initialized using PCA, and training was run for 300 epochs (corresponding to approximately 3 hours on a 16-core Azure machine)”), first features of different datasets (Abstract, “experiments performed in hundreds of different datasets”, P. 2, ¶3, “experiments already performed across different datasets D = {D1; : : : ;DD}”, P. 4, 3, ¶2, “This gives us a matrix
PNG
media_image1.png
35
156
media_image1.png
Greyscale
summarizing the performance of each pipeline in each dataset.”, “we are seeking a low rank decomposition such that … where Q is the dimensionality of the latent space”, “both X and W are unknown and must be inferred”) and second visualization latent-space features of different computation pipelines represented in a second two-dimensional latent space (Fig. 1, “Two-dimensional embedding of 5,000 ML pipelines across 576 OpenML datasets. Each point corresponds to a pipeline”, P. 2, ¶4, “we refer to the combination of pre-processing method, machine learning model and their hyperparameters as an ML pipeline”, ¶5, “embeds different pipelines in a latent space based on their performance across different datasets”, P. 7, 4.2, “The latent space was initialized using PCA, and training was run for 300 epochs (corresponding to approximately 3 hours on a 16-core Azure machine)”);
modeling, by the at least one computing device, dataset-pipeline interactions between the different datasets and the different computation pipelines based on the first features and the second visualization latent-space features (P. 2, ¶5, “the problem of predicting the performance of ML pipelines on a new dataset can be cast as a collaborative filtering problem that can be solved with probabilistic matrix factorization techniques”, P. 4, “we develop a method that can draw information from all of the datasets for which experiments are available, whether they are immediately related (e.g. a smaller version of the current dataset) or not. The idea behind our approach is that if two datasets have similar (i.e. correlated) results for a few pipelines, it’s likely that the remaining pipelines will produce results that are similar as well. This is somewhat reminiscent of a collaborative filtering problem … the task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”); and
learning, by the at least one computing device, relationships between the first features and the second visualization latent-space features based on modeling the dataset-pipeline interactions (P. 4, “we develop a method that can draw information from all of the datasets for which experiments are available, whether they are immediately related (e.g. a smaller version of the current dataset) or not. The idea behind our approach is that if two datasets have similar (i.e. correlated) results for a few pipelines, it’s likely that the remaining pipelines will produce results that are similar as well. This is somewhat reminiscent of a collaborative filtering problem … the task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”, P. 8, “our model is able to effectively capture most of this information in a completely unsupervised fashion, just by observing the sparse pipelines-dataset matrix Y”, P. 8, “Similarity between different pipelines is induced by having correlated performance across multiple datasets”).
D1 does not teach first visualization latent-space
D2 teach a method to analyze computation pipelines and datasets, the method comprising (¶1, “Computer-implemented methods are provided for selecting preferred machine learning pipelines for processing datasets, together with systems and computer program products implementing such methods “, ¶3, “One aspect of the present disclosure provides a computer-implemented method for selecting preferred machine learning pipelines for processing new datasets”):
learning, by at least one computing device, first visualization latent-space features of different datasets represented in a first two-dimensional latent space (Figs. 5-6, ¶4, “datasets are embedded based on a notion of “pairwise expected regret”“, “Methods embodying this aspect of the disclosure exploit a latent space, in which datasets are embedded”, ¶52, “The above process embeds processed datasets in the latent space χ”, ¶57, “FIG. 5 is a simplified schematic illustrating embedding of datasets in the latent space χ. Datasets are embedded in this space such that the distance … corresponds to an expected value … of the regret …”, ¶59), features of different computation pipelines (Fig. 7, ¶3, “for a plurality of machine learning pipelines and a plurality N of datasets previously-processed by the pipelines, storing rating values each rating performance of a pipeline for a dataset”, ¶60);
modeling, by the at least one computing device, dataset-pipeline interactions between the different datasets and the different computation pipelines (Fig. 3, ¶3, “storing rating values each rating performance of a pipeline for a dataset”, ¶53, “Pipeline processing typically involves not only training but also testing performance of the trained model, for example on a holdout set, i.e. a subset of the original dataset reserved for testing inference performance Based on results of these performance tests, pipeline selector 25 can identify the best-performing pipeline for the user application”) based on the first visualization latent-space features (¶4, “pairwise expected regret, which determines distance between each pair of datasets in the latent space, captures an expected drop in performance rating for a pipeline on a dataset when that pipeline is selected based on performance ratings for another dataset“);
learning, by the at least one computing device, relationships between the first visualization latent-space features (¶4, “By clustering datasets in this space, the datasets can be grouped into “neighborhoods” for which a number of representative datasets, one from each neighborhood, can be identified. New datasets are likely to be close to these representative datasets in the latent space. By selecting preferred pipelines based on rating values of representative datasets, the selected pipelines are likely to be rated highly for new datasets”) and based on modeling the dataset-pipeline interactions (¶52, “The above process embeds processed datasets in the latent space χ based on the notion of a pairwise expected regret which reflects drop in performance rating for a pipeline on a dataset when that pipeline is selected based on ratings for another dataset”, ¶53, “Since one of the representative datasets will be closest to the new dataset in latent space χ, and hence incur the least expected regret, this increases the likelihood of selecting a high-performance pipeline for the new dataset”, ¶54, “These techniques allow approximation of the representative dataset which is closest to a new dataset in the latent space χ, as well as controlled exploration of pipeline recommendations”, ¶55, “a pipeline is selected as a random variable from a probability distribution conditioned on the rating values for that dataset”, ¶58, “Since rating Ri,k is a monotonically decreasing function of regret G(k; ui), the monotonicity also holds true for the expectations. That is, smaller expected regret values Ĝi,j, and hence smaller distances di,j, correspond to higher expected values for the ratings”).
D1 and D2 are analogous art to the claimed invention because they are from a similar field of endeavor of AutoML/AutoAI systems for selecting machine learning pipelines for datasets based on prior performance of pipelines on datasets. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1 resulting in resolutions as disclosed by D2 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1 as described above to provide an effective and elegant solution to the problem of selecting preferred ML pipelines for new datasets in AutoAI which leads to more efficient training of ML models, better performing models, and improved operation of user applications for these models. Also to offer improved control of the exploitation-exploration trade-off during automated pipeline selection (D2, ¶4).
With regard to Claim 10,
Claim 10 is similar in scope to claim 1; therefore it is rejected under similar rationale.
Claims 2-3, 8 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems” published 4/2022 hereinafter D3.
With regard to Claim 2,
D1-D2 teach the method of claim 1.
D1-D2 does not teach wherein learning the first visualization latent-space features and the second visualization latent-space features comprises: implementing, by the at least one computing device, two structurally symmetric variational autoencoders in parallel to respectively learn the first visualization latent-space features in the first two-dimensional latent space and the second visualization latent-space features in the second two-dimensional latent space.
D3 teach wherein learning the first visualization latent-space features and the second visualization latent-space features comprises (Abstract, P. 1, “This paper proposes a deep learning model specifically designed to display the existing relations among users, items, and both users and items”, “bidimensional and three dimensional representations of users and items. The proposed neural model incorporates variational embedding stages”):
implementing, by the at least one computing device, two structurally symmetric variational autoencoders in parallel (P. 3, ¶2, “From the explained research, this paper proposes an innovative deep learning model that incorporates two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”) to respectively learn the first visualization latent-space features in the first two-dimensional latent space and the second visualization latent-space features in the second two-dimensional latent space (P. 3, ¶2, “to make it possible to draw bi- or three-dimensional graphs of user and item samples. The accuracy loss caused by the small embedding sizes (two or three neurons each embedding)”, P. 10, “embedding sizes to two neurons (obtaining two-dimensional graphs) or to three neurons (obtaining three-dimensional graphs)”).
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of collaborative filtering recommender style learning to select or predict suitable items for a target. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D2 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve learning of two dataset pipeline latent space and better model their interactions for pipeline recommendation, thereby improving pipeline performance prediction accuracy (D3, Abstract, “Current collaborative filtering machine learning models are designed to improve prediction accuracy”).
With regard to Claim 3,
D1-D2 teach the method of claim 1, wherein modeling the dataset-pipeline interactions (P. 4, 3, “given N machine learning pipelines and D datasets, we train each pipeline on part of each dataset and we evaluate it on a holdout set. This gives us a matrix
PNG
media_image1.png
35
156
media_image1.png
Greyscale
summarizing the performance of each pipeline in each dataset.”, “task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”) comprises:
to predict performance metrics of the different computation pipelines with respect to the different datasets (P. 4, 3, “given N machine learning pipelines and D datasets, we train each pipeline on part of each dataset and we evaluate it on a holdout set. This gives us a matrix
PNG
media_image1.png
35
156
media_image1.png
Greyscale
summarizing the performance of each pipeline in each dataset.”, “task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”) based on the first visualization latent-space features and the second visualization latent-space features.
D1-D2 does not teach implementing, by the at least one computing device, a neural collaborative filtering network
D3 teach implementing, by the at least one computing device, a neural collaborative filtering network (Fig. 4, “collaborative filter prediction”, P. 3, 2., “The current deep CF state of the art includes two remarkable neural models: DeepMF (Figure 1a) and NCF (Figure 1b).”, “the NCF model (Figure 1b) incorporates an MLP that non-linearly combines factors of the user and the item, returning scalar regression values (predictions)”) to predict performance metrics of the different computation pipelines with respect to the different datasets ((P. 3, 2., “the NCF model … returning scalar regression values (predictions)”, P. 6, Code 1, “similar = Lambda(euclidean)([movie_vec, user_vec]) var_eucl_pred = Model([user_input, movie_input], similar)”) based on the first visualization latent-space features and the second visualization latent-space features (P. 1, “The innovative model incorporates embedding layers of small (representable) sizes, variational layers to improve the latent space”, P. 3, 2. ,” DeepMF (Figure 1a) and NCF (Figure 1b) models provide two embedding layers: the first codes users and the second codes items” P. 3, ¶2, “variational layers to improve the latent space”, P. 6, “The parallel user and item flows (orange and blue ones) provide both the user variational vector and the item variational vector … they would be merged … “Euclidean layer’”, P. 3, ¶2, “two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”)
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of collaborative filtering recommender style learning to select or predict suitable items for a target. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D3 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve learning of two dataset pipeline latent space and better model their interactions for pipeline recommendation, thereby improving pipeline performance prediction accuracy (D3, Abstract, “Current collaborative filtering machine learning models are designed to improve prediction accuracy”).
With regard to Claim 8,
D1-D2 teach the method of claim 1, further comprising:
the first visualization latent-space features in the first two- dimensional latent space based on first visualization latent-space representations that correspond to and represent the different datasets in the first two-dimensional latent space (D2, Figs. 5-6, ¶4, “Methods embodying this aspect of the disclosure exploit a latent space, in which datasets are embedded”, ¶57, “FIG. 5 is a simplified schematic illustrating embedding of datasets in the latent space χ. Datasets are embedded in this space such that the distance du from a dataset ui to a dataset uj corresponds to an expected value Ĝi,j of the regret”, ¶2, “to make it possible to draw bi- or three-dimensional graphs of user and item samples. The accuracy loss caused by the small embedding sizes (two or three neurons each embedding)”, P. 10, “embedding sizes to two neurons (obtaining two-dimensional graphs) or to three neurons (obtaining three-dimensional graphs)”) ; and
the second visualization latent-space features in the second two-dimensional latent space based on second visualization latent-space representations that correspond to and represent the different computation pipelines in the second two-dimensional latent space (D1, Fig. 1, “Two-dimensional embedding of 5,000 ML pipelines across 576 OpenML datasets. Each point corresponds to a pipeline”, P. 2, ¶4, “we refer to the combination of pre-processing method, machine learning model and their hyperparameters as an ML pipeline”, ¶5, “embeds different pipelines in a latent space based on their performance across different datasets”).
D1-D2 does not teach learning, by the at least one computing device using a first of two structurally symmetric variational autoencoders , learning, by the at least one computing device using a second of the two structurally symmetric variational autoencoders.
D3 teach learning, by the at least one computing device using a first of two structurally symmetric variational autoencoders (P. 6, code 1, code 1 show user variational branch of parallel branches including user embedding, user embedding mean, user embedding variance, and lambda sampling to produce user variational vector, P. 6, “The parallel user and item flows (orange and blue ones) provide both the user variational vector and the item variational vector … they would be merged … “Euclidean layer’”, P. 3, ¶2, “two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”), learning, by the at least one computing device using a second of the two structurally symmetric variational autoencoders (P. 6, code 1, code 1 show movie/item variational branch of parallel branches including movie embedding, movie embedding mean, movie embedding variance, and lambda sampling to produce movie variational vector, P. 6, Code. 1, P. 6, “The parallel user and item flows (orange and blue ones) provide both the user variational vector and the item variational vector … they would be merged … “Euclidean layer’”, P. 3, ¶2, “two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”).
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of collaborative filtering recommender style learning to select or predict suitable items for a target. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D3 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve learning of two dataset pipeline latent space and better model their interactions for pipeline recommendation, thereby improving pipeline performance prediction accuracy (D3, Abstract, “Current collaborative filtering machine learning models are designed to improve prediction accuracy”).
With regard to Claim 11,
Claim 11 is similar in scope to claim 2; therefore it is rejected under similar rationale.
With regard to Claim 12,
Claim 12 is similar in scope to claim 3; therefore it is rejected under similar rationale.
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “Neural Collaborative Filtering” published 4/2017 hereinafter D4.
With regard to Claim 4,
D1-D2 teach the method of claim 1, wherein modeling the dataset-pipeline interactions (P. 4, 3, “given N machine learning pipelines and D datasets, we train each pipeline on part of each dataset and we evaluate it on a holdout set. This gives us a matrix
PNG
media_image1.png
35
156
media_image1.png
Greyscale
summarizing the performance of each pipeline in each dataset.”, “task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”).
D1-D2 does not explicitly teach implementing, by the at least one computing device, a neural collaborative filtering network to model the dataset-pipeline interactions using a neural collaborative filtering process comprising generalized matrix factorization of a recommendation matrix generated by the neural collaborative filtering network and multi-layer projection of the first visualization latent-space features and the second visualization latent-space features.
D4 teach wherein modeling the dataset-pipeline interactions comprises:
implementing, by the at least one computing device, a neural collaborative filtering network (D4, P. 1, Abstract, “By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network based Collaborative Filtering”, “a multi-layer perceptron to learn the user–item interaction function”, P. 3, Col. 2, ¶2, “user embedding and item embedding are then fed into a multi-layer neural architecture, which we term as neural collaborative filtering layers, to map the latent vectors to prediction scores”, P. 3, 3.1, “To permit a full neural treatment of collaborative filtering, we adopt a multi-layer representation to model a user–item interaction yui as shown in Figure 2”) to model the dataset-pipeline interactions using a neural collaborative filtering process (P. 1, Abstract, “a multi-layer perceptron to learn the user–item interaction function”, P. 3, Col. 2, ¶2, “user embedding and item embedding are then fed into a multi-layer neural architecture, which we term as neural collaborative filtering layers, to map the latent vectors to prediction scores”) comprising generalized matrix factorization of a recommendation matrix generated by the neural collaborative filtering network (Fig. 3, P. 2, “2.1, “We define the user–item interaction matrix Y ∈ RM×N from users’implicit feedback …”, P. 4, 3.2, “We term it as GMF, short for Generalized Matrix Factorization.”, P.1, Abstract, “NCF is generic and can express and generalize matrix factorization under its framework” P. 5, Col. 1, “we allow GMF and MLP to learn separate embeddings, and combine the two models by concatenating their last hidden layer”, NCF/GMF predicts entries of user item recommendation matrix applied to D1-D2 dataset pipeline setting , the same process generates predicted dataset pipeline recommendation performance score, which form the recommendation matrix as required in the claim limitation) and multi-layer projection of the first visualization latent- space features and the second visualization latent-space features (Fig. 2, Fig. 3, P. 3, 3.1, “Above the input layer is the embedding layer; it is a fully connected layer that projects the sparse representation to a dense vector. The obtained user (item) embedding can be seen as the latent vector for user (item) in the context of latent factor model. The user embedding and item embedding are then fed into a multi-layer neural architecture, which we term as neural collaborative filtering layers, to map the latent vectors to prediction scores.”, P. 4, 3.3, “we propose to add hidden layers on the concatenated vector, using a standard MLP to learn the interaction between user and item latent features.”, Eq. (10), P. 4, 3.4, “So far we have developed two instantiations of NCF — GMF that applies a linear kernel to model the latent feature interactions, and MLP that uses a non-linear kernel”, P. 5, Col. 1, “we allow GMF and MLP to learn separate embeddings, and combine the two models by concatenating their last hidden layer”).
D1-D2 and D4 are analogous art to the claimed invention because they are from a similar field of endeavor of using collaborative filtering factorization concepts to predict interaction between two types of entities. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D4 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve recommendation performance as using deeper neural network layers provides better recommendation performance (D4 P. 1, Abstract, ¶3).
With regard to Claim 13,
Claim 13 is similar in scope to claim 4; therefore it is rejected under similar rationale.
Claims 5, 14 and 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “A 3D Item Space Visualization for Presenting and Manipulating User Preferences in Collaborative Filtering” published 3/2017 hereinafter D5.
With regard to Claim 5,
D1-D2 teach the method of claim 1.
D1-D2 does not teach generating, by the at least one computing device, a visual representation of the relationships and the dataset-pipeline interactions, the visual representation comprising:
latitude and longitude data that are indicative of the relationships; and altitude data that are indicative of the dataset-pipeline interactions.
D5 teach generating, by the at least one computing device, a visual representation of the relationships and the dataset-pipeline interactions (P. 1, Abstract, “we suggest a 3D map-based visualization of the entire item space in which we position and present sample items along with recommendations. The map is produced by mapping latent factors obtained from Collaborative Filtering data onto a 2D surface through Multidimensional Scaling.”), the visual representation comprising:
latitude and longitude data that are indicative of the relationships (P. 2, Col. 1, ¶6, “map the resulting high dimensional latent factor model onto a two-dimensional surface in which all items are positioned with respect to their similarities”, P. 4, “MDS to calculate two-dimensional coordinates for all items. Using these coordinates, the resulting map visualization positions items based on their similarities (Figure 1, 1b)”); and
altitude data that are indicative of the dataset-pipeline interactions (P. 1, Abstract, “areas that contain items relevant with respect to the current user’s preferences are shown as elevations on the map, areas of low interest as valleys”, P. 2, Col. 1, ¶6, “we additionally exploit the third dimension. Therefore, we use the MF predictions for the current user and all items in order to form a landscape where elevations represent areas with high estimated ratings while valleys indicate lower relevance”, P. 5, Col. 2, ¶2, “we linearly map the prediction for every item onto a height value, and consequently set the surface elevation at the item’s respective position to this value”).
D1-D2 and D5 are analogous art to the claimed invention because they are from a similar field of endeavor of computer generated visual representation of multi-dimensional data relationships for user interpretation and interaction. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D4 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to automatically generate personalized pipelines suggestions with a simple visual representation that help users to understand why certain pipelines are recommended and which portions of the pipelines space are covered by the recommendations, and to provide much broader means to influence the process of generating results (D5, Abstract).
With regard to Claim 14,
Claim 14 is similar in scope to claim 5; therefore it is rejected under similar rationale.
With regard to Claim 16,
Claim 16 is similar in scope to claim 1 and claim 5; therefore it is rejected under similar rationale.
With regard to Claim 17,
D1-D2-D5 teach the method of claim 16, wherein generating the visual representation comprises:
constructing, by the at least one computing device, a two-dimensional meshgrid of the different datasets or the different computation pipelines in the first two-dimensional latent space or the second two-dimensional latent space, respectively (D1, (Fig. 1, “Two-dimensional embedding of 5,000 ML pipelines across 576 OpenML datasets. Each point corresponds to a pipeline”, P. 2, ¶4, “we refer to the combination of pre-processing method, machine learning model and their hyperparameters as an ML pipeline”, ¶5, “embeds different pipelines in a latent space based on their performance across different datasets”, D2, Figs. 5-6, ¶4, “datasets are embedded based on a notion of “pairwise expected regret”“, “Methods embodying this aspect of the disclosure exploit a latent space, in which datasets are embedded”, ¶52, “The above process embeds processed datasets in the latent space χ”, ¶57, “FIG. 5 is a simplified schematic illustrating embedding of datasets in the latent space χ. Datasets are embedded in this space such that the distance … corresponds to an expected value … of the regret …”, ¶59, D5, “P. 1, Abstract, “we suggest a 3D map-based visualization of the entire item space in which we position and present sample items along with recommendations. The map is produced by mapping latent factors obtained from Collaborative Filtering data onto a 2D surface through Multidimensional Scaling.”, P. 2, Col. 1, ¶6, “map the resulting high dimensional latent factor model onto a two-dimensional surface in which all items are positioned with respect to their similarities”, P. 4, “MDS to calculate two-dimensional coordinates for all items. Using these coordinates, the resulting map visualization positions items based on their similarities (Figure 1, 1b)””). The same motivation to combine for claim 16 equally applies for current claim.
With regard to Claim 18,
D1-D2-D5 teach the method of claim 17, further comprising:
determining, by the at least one computing device, a three-dimensional predicted
visualization profile for each meshgrid cell in the two-dimensional meshgrid (D5, P. 1, Abstract, “areas that contain items relevant with respect to the current user’s preferences are shown as elevations on the map, areas of low interest as valleys”, P. 2, Col. 1, ¶6, “we additionally exploit the third dimension. Therefore, we use the MF predictions for the current user and all items in order to form a landscape where elevations represent areas with high estimated ratings while valleys indicate lower relevance”, P. 5, Col. 2, ¶2, “we linearly map the prediction for every item onto a height value, and consequently set the surface elevation at the item’s respective position to this value”); and
generating, by the at least one computing device, a colormap for each meshgrid cell of the two-dimensional meshgrid based on the three-dimensional predicted visualization profile, the colormap being indicative of predicted performance data of one of the different computation pipelines with respect to one of the different datasets (D5, P. 3, Col. 1, “areas of interest that include the recommendations can still be highlighted by color, similar to a heat map”, P. 6, Col. 1, Interaction concept, “we color the surface to resemble a topographical map. Therefore, we use a function that assigns colors to particular levels of elevation while ensuring smooth transitions between them”, Fig. 2, Fig. 3, Fig. 4, “Figure 2. Screenshot of our demonstrator: Working area (A) visualizing the item space as a quadratic map that includes movie posters depicting the automatically chosen sample items and represents the user’s preferences by the surface elevation; recommended items (B), which are also shown inside the landscape as posters highlighted by a magenta-colored margin; detail information on the currently selected movie (C); and a palette (D) of available interaction tools (in this example, the Raise/Dig-tool is selected, which appears at the position of the cursor in the lower middle part of the screen).”, P. 1, Abstract, “areas that contain items relevant with respect to the current user’s preferences are shown as elevations on the map, areas of low interest as valleys”, P. 2, Col. 1, ¶6, “we additionally exploit the third dimension. Therefore, we use the MF predictions for the current user and all items in order to form a landscape where elevations represent areas with high estimated ratings while valleys indicate lower relevance”, P. 5, Col. 2, ¶2, “we linearly map the prediction for every item onto a height value, and consequently set the surface elevation at the item’s respective position to this value”). The same motivation to combine for claim 16 equally applies for current claim.
With regard to Claim 19,
D1-D2-D5 teach the method of claim 18, further comprising:
arranging, by the at least one computing device, meshgrid cells in the two-dimensional meshgrid based at least one of the relationships or the performance data, wherein subsets of the meshgrid cells having at least one of similar relationships or similar performance data are arranged adjacent to one another (P. 2, Col. 1, ¶6, “map the resulting high dimensional latent factor model onto a two-dimensional surface in which all items are positioned with respect to their similarities”, P. 4, “MDS to calculate two-dimensional coordinates for all items. Using these coordinates, the resulting map visualization positions items based on their similarities (Figure 1, 1b)”). The same motivation to combine for claim 16 equally applies for current claim.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “LEARNING META-FEATURES FOR AUTOML” published on 3/2022 hereinafter D6 in view of “AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space” published on 8/2020 hereinafter D7.
With regard to Claim 6,
D1-D2 teach the method of claim 1.
D1-D2 does not teach generating, by the at least one computing device, meta-data vectors respectively corresponding to the different datasets, the meta-data vectors being generated based on respective summary statistics data of the different datasets; and
generating, by the at least one computing device, embedding vectors respectively corresponding to the different computation pipelines based on data indicative of different pipeline component candidates of the different computation pipelines.
D6 teach generating, by the at least one computing device, meta-data vectors respectively corresponding to the different datasets (P. 1, Introduction, “Early approaches have been investigating the use of general performance models (Rice, 1976), estimating a priori the performance of any algorithm on any problem instance, where each problem instance is described by a vector of so-called meta-features, and the performance model is learned in this meta-feature space”), the meta-data vectors being generated based on respective summary statistics data of the different datasets (P. 2, 2, “Most ML meta-features … have been manually designed to describe supervised datasets based on descriptive statistics, information theory (quantifying relationships among features/labels), geometrical structure of the dataset, and landmarking”, P. 4, Principle, “The basic representation x ε IRD of a dataset reports the values of the D manually designed meta-features for this dataset. By construction, it can be cheaply computed for any dataset”).
D1-D2 and D6 are analogous art to the claimed invention because they are from a similar field of endeavor of AutoML field to use prior datasets for selecting ML algorithms or pipelines for a dataset by using information learned from prior datasets. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D6 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to achieve AutoML with a decent performance vs cost trade-off and to define a reliable topology on the dataset space, such that two datasets are close if the best hyper-parameter configurations for these datasets are close. Such a topology would support an inexpensive and efficient AutoML strategy selecting the best hyper-parameter configurations of the nearest neighbor(s) of the current dataset (D6 P. 1-2).
D1-D2-D6 does not explicitly teach generating, by the at least one computing device, embedding vectors respectively corresponding to the different computation pipelines based on data indicative of different pipeline component candidates of the different computation pipelines.
D7 teach generating, by the at least one computing device, embedding vectors respectively corresponding to the different computation pipelines (P. 3, 3.3, “Each factor matrix can thus be viewed as embedding the corresponding dataset or pipeline component, with pipeline embeddings as columns of Y …”, “X … and Y … are dataset and pipeline embeddings, respectively”) based on data indicative of different pipeline component candidates of the different computation pipelines (P. 2, Col. 2, “Pipeline component. A pipeline component is a model or model type. Examples include missing entry imputers, dimensionality reducers, supervised learners, and data visualizers”, “components in this paper: Data imputer … Encoder … Standardizer … Dimensionality reducer … Estimator …”, P. 2, Col. 1, ¶3, “In this work, we build pipeline embeddings by fitting a tensor decomposition to the (incompletely observed) tensor of pipeline performance on a set of training datasets”, P. 2, Col. 2, “Pipeline errors on training datasets form an error tensor, which we denote as E. In our experiments, E is an order-6 tensor, with 6 modes corresponding to datasets, imputers, encoders, standardizers, dimensionality reducers and estimators, respectively. The (i1, i2, . . . , i6)-th entry of E is the error of the pipeline formed by composing the i2-th imputer, i3-th encoder, i4-th standardizer, i5-th dimensionality reducer, and i6-th estimator and evaluating this pipeline on the i1-th dataset).
D1-D2-D6 and D7 are analogous art to the claimed invention because they are from a similar field of endeavor of AutoML because both model the performance of different ML pipelines on different datasets using factorization based representation to select high performing pipelines. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2-D6 resulting in resolutions as disclosed by D7 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2-D6 as described above better capture the structure of ML pipelines and navigate the large combinatorial pipeline space more efficiently by using surrogate model that enables efficient search through the pipeline space and makes predictions that guide the search for pipelines without the need for many model fits (D7, P. 2, Col. 1, ¶¶1-4).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems” published 4/2022 hereinafter D3 in view of “LEARNING META-FEATURES FOR AUTOML” published on 3/2022 hereinafter D6.
With regard to Claim 7,
D1-D2 teach the method of claim 1, further comprising:
first visualization latent-space representations of the different datasets in the first two-dimensional latent space (D2, Figs. 5-6, ¶4, “Methods embodying this aspect of the disclosure exploit a latent space, in which datasets are embedded”, ¶57, “FIG. 5 is a simplified schematic illustrating embedding of datasets in the latent space χ. Datasets are embedded in this space such that the distance du from a dataset ui to a dataset uj corresponds to an expected value Ĝi,j of the regret”); and
generating, by the at least one computing device using a second of the two structurally symmetric variational autoencoders, second visualization latent-space representations of the different computation pipelines in the second two-dimensional latent space (D1, Fig. 1, “Two-dimensional embedding of 5,000 ML pipelines across 576 OpenML datasets. Each point corresponds to a pipeline”, P. 2, ¶4, “we refer to the combination of pre-processing method, machine learning model and their hyperparameters as an ML pipeline”, ¶5, “embeds different pipelines in a latent space based on their performance across different datasets”) based on embedding vectors respectively corresponding to the different computation pipelines (D1, P. 2, ¶3, “experiments already performed across different datasets D = {D1; : : : ;DD}”, P. 4, 3, ¶2, “This gives us a matrix
PNG
media_image1.png
35
156
media_image1.png
Greyscale
summarizing the performance of each pipeline in each dataset.”, “we are seeking a low rank decomposition such that … where Q is the dimensionality of the latent space … where xn is a row of the latent variables X and yn is a row of measured performances for pipeline n”, Eq(1)).
D1-D2 does not explicitly teach generating, by the at least one computing device using a first of two structurally symmetric variational autoencoders.
D3 teach generating, by the at least one computing device using a first of two structurally symmetric variational autoencoders (P. 6, Code. 1, P. 6, “The parallel user and item flows (orange and blue ones) provide both the user variational vector and the item variational vector … they would be merged … “Euclidean layer’”, P. 3, ¶2, “two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”).
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of collaborative filtering recommender style learning to select or predict suitable items for a target. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D3 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve learning of two dataset pipeline latent space and better model their interactions for pipeline recommendation, thereby improving pipeline performance prediction accuracy (D3, Abstract, “Current collaborative filtering machine learning models are designed to improve prediction accuracy”).
D1-D2-D3 does not teach based on meta-data vectors respectively corresponding to the different datasets.
D6 teach based on meta-data vectors respectively corresponding to the different datasets (P. 4, Principle, “The basic representation x ε IRD of a dataset reports the values of the D manually designed meta-features for this dataset. By construction, it can be cheaply computed for any dataset”, P. 2, 2, “Most ML meta-features … have been manually designed to describe supervised datasets based on descriptive statistics, information theory (quantifying relationships among features/labels), geometrical structure of the dataset, and landmarking”)
D1-D2 and D6 are analogous art to the claimed invention because they are from a similar field of endeavor of AutoML field to use prior datasets for selecting ML algorithms or pipelines for a dataset by using information learned from prior datasets. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D6 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to achieve AutoML with a decent performance vs cost trade-off and to define a reliable topology on the dataset space, such that two datasets are close if the best hyper-parameter configurations for these datasets are close. Such a topology would support an inexpensive and efficient AutoML strategy selecting the best hyper-parameter configurations of the nearest neighbor(s) of the current dataset (D6 P. 1-2).
Claims 9 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “Deep Variational Embedding Representation on Neural Collaborative Filtering Recommender Systems” published 4/2022 hereinafter D3 in view of Aggarwal et al. [US20220019888 A1, hereinafter Agg].
With regard to Claim 9,
D1-D2 teach the method of claim 1, further comprising:
to learn the relationships (D1, P. 4, “we develop a method that can draw information from all of the datasets for which experiments are available, whether they are immediately related (e.g. a smaller version of the current dataset) or not. The idea behind our approach is that if two datasets have similar (i.e. correlated) results for a few pipelines, it’s likely that the remaining pipelines will produce results that are similar as well. This is somewhat reminiscent of a collaborative filtering problem … the task of predicting the performance of any of them on a new dataset can be cast as a matrix factorization problem”).
D1-D2 does not teach training, by the at least one computing device, a joint variational autoencoder neural collaborative filtering network to learn the relationships using a hybrid loss function that drives the joint variational autoencoder neural collaborative filtering network toward achieving at least one of: a defined cluster-wised ranking prediction accuracy; a defined visualization latent-space representation generation accuracy in the first two- dimensional latent space and the second two-dimensional latent space; or a defined visualization profile continuity and smoothness of a visual representation of the relationships and the dataset-pipeline interactions.
D3 teach training, by the at least one computing device, a joint variational autoencoder neural collaborative filtering network to learn the relationships ( (P. 3, ¶2, “From the explained research, this paper proposes an innovative deep learning model that incorporates two embedding layers: one for code users and the other for code items”, “Both user and item embeddings will be followed by their own Gaussian variational layers whose parameter values are learned in the whole neural model”, P. 6, ¶3, “The parallel user and item flows (orange and blue ones) provide both the user variational vector”, P. 2-3, ¶4, “mimics the underlying VAE operative … using VAE, the latent space is enriched, and samples are spread”) that drives the joint variational autoencoder neural collaborative filtering network toward achieving at least one of:
a defined cluster-wised ranking prediction accuracy;
a defined visualization latent-space representation generation accuracy in the first two-dimensional latent space and the second two-dimensional latent space (Abstract, P. 1, “This paper proposes a deep learning model specifically designed to display the existing relations among users, items, and both users and items”, P. 3, ¶2, “to make it possible to draw bi- or three-dimensional graphs of user and item samples. The accuracy loss caused by the small embedding sizes (two or three neurons each embedding)”, P. 10, “embedding sizes to two neurons (obtaining two-dimensional graphs) or to three neurons (obtaining three-dimensional graphs)”, P. 7, Table 2, “we use embedding sizes: 2 (two-dimensional representation) or 3 (three-dimensional representation), whereas the usual implementation sizes range from 5 to 10. Our first experiment tests the accuracy loss when small embedding sizes are set. For each tested dataset (Table 1), we obtain the Mean Absolute Error (MAE) by setting embedding sizes = {2, 3, 5, 10}. Table 2 shows the MAE results, as well as the achieved accuracy percentage”); or a defined visualization profile continuity and smoothness of a visual representation of the relationships and the dataset-pipeline interactions.
D1-D2 and D3 are analogous art to the claimed invention because they are from a similar field of endeavor of collaborative filtering recommender style learning to select or predict suitable items for a target. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2 resulting in resolutions as disclosed by D3 with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2 as described above to improve learning of two dataset pipeline latent space and better model their interactions for pipeline recommendation, thereby improving pipeline performance prediction accuracy (D3, Abstract, “Current collaborative filtering machine learning models are designed to improve prediction accuracy”).
D1-D2-D3 does not each using a hybrid loss function.
Agg teach training, by the at least one computing device, using a hybrid loss function (Fig. 3, Abstract, “A single unified machine learning model (e.g., a neural network) is trained to perform both supervised event predictions and unsupervised time-varying clustering for a sequence of events (e.g., a sequence representing a user behavior) using sequences of events for multiple users using a combined loss function“). that drives the joint variational autoencoder neural collaborative filtering network toward achieving (¶25, “multiple loss functions are used for training the neural network … the various loss functions are combined into a combined loss function. The combined loss function is then used during the training phase to optimize the unified model … so that next event predictions (including event type and time of occurrence) more closely correspond to corresponding ground truth values as the loss is minimized and so that clustering output is interpretable or otherwise useful “).
D1-D2-D3 and Agg are analogous art to the claimed invention because they are from a similar field of endeavor of using machine learning models to predict outcomes from learned vector representations. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2-D3 resulting in resolutions as disclosed by Agg with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2-D3 as described above to optimize the model so that predicted pipeline performance values more closely correspond to ground truth performance values as the loss is minimized, and so that the learned clustering output is interpretable or otherwise useful (Agg, ¶25).
With regard to Claim 15,
Claim 15 is similar in scope to claim 9; therefore it is rejected under similar rationale.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over “Probabilistic Matrix Factorization for Automated Machine Learning” Published 2018 [hereinafter D1] in view of Gorbach et al. [US20220171985A1, hereinafter D2] in view of “A 3D Item Space Visualization for Presenting and Manipulating User Preferences in Collaborative Filtering” published 3/2017 hereinafter D5 in view of Matthew et al. [US 20170026572 A1, hereinafter Matthew].
With regard to Claim 20,
D1-D2-D5 teach the method of claim 19, further comprising:
projecting, by the at least one computing device, the two-dimensional meshgrid to a three-dimensional space (D5, P. 1, Abstract, “areas that contain items relevant with respect to the current user’s preferences are shown as elevations on the map, areas of low interest as valleys”, P. 2, Col. 1, ¶6, “we additionally exploit the third dimension. Therefore, we use the MF predictions for the current user and all items in order to form a landscape where elevations represent areas with high estimated ratings while valleys indicate lower relevance”, P. 5, Col. 2, ¶2, “we linearly map the prediction for every item onto a height value, and consequently set the surface elevation at the item’s respective position to this value”); and
generating, by the at least one computing device, visual representation of the relationships and the performance data based on projecting the two-dimensional meshgrid to the three-dimensional space (D5, P. 2, Col. 1, ¶6, “map the resulting high dimensional latent factor model onto a two-dimensional surface in which all items are positioned with respect to their similarities”, P. 4, “MDS to calculate two-dimensional coordinates for all items. Using these coordinates, the resulting map visualization positions items based on their similarities (Figure 1, 1b)”, P. 1, Abstract, “areas that contain items relevant with respect to the current user’s preferences are shown as elevations on the map, areas of low interest as valleys”, P. 2, Col. 1, ¶6, “we additionally exploit the third dimension. Therefore, we use the MF predictions for the current user and all items in order to form a landscape where elevations represent areas with high estimated ratings while valleys indicate lower relevance”, P. 5, Col. 2, ¶2, “we linearly map the prediction for every item onto a height value, and consequently set the surface elevation at the item’s respective position to this value”).
D1-D2-D5 does not explicitly teach a sphere-shaped visual representation.
Matthew teach projecting, by the at least one computing device, the two-dimensional meshgrid to a three-dimensional space (Fig. 13-14, ¶63, “next step in determining the panel model is to compute the plane grid. The method of computing the plane grid”, “ECU 110 calculates a set of grid points on the local plane 805”, “A local plane plaid grid is given by: {x.sub.LTP,y.sub.LTP}=meshgrid(x.sub.LTP,y.sub.LTP) (30) where mesghrid is a standard MATLAB function, and x.sub.LTP and y.sub.LTP are matrices that store the grid points“); and
generating, by the at least one computing device, a sphere-shaped visual representation of the relationships and the performance data based on projecting the two-dimensional meshgrid to the three-dimensional space (¶44, “Once the region of interest is defined, the panel transform converts a portion of the omnidirectional image into the rectilinear image with the methods presented herein. The transformation of the portion of the image is enabled by first mapping a relationship between the region of interest in the omnidirectional image and a perspective view of that region of interest”, ¶7, “ on a unit sphere in an optical coordinate system”, “receives a set of parameters defining a local plane tangent to the unit sphere”, ¶56, “local plane 805 tangent to the viewing sphere 710 is illustrated in FIG. 8”, “¶57, “The panel model specifies an area of interest tangent to the viewing sphere 710 in reference to the OCS, and thereby also specifies an area of interest on the viewing sphere 710”, ¶76, “The OCS point is mapped to the viewing sphere 710”, claim 5).
D1-D2-D5 and Matthew are analogous art to the claimed invention because they are from a similar field of endeavor of computer generated visual representation of multi-dimensional data relationships for user interpretation and interaction. Thus, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify D1-D2-D5 resulting in resolutions as disclosed by Matthew with a reasonable expectation of success.
One of ordinary skill in the art would be motivated to modify D1-D2-D3 as described above as presenting the learned relationships and predicted data in a sphere shaped 3D would provide the user with additional intuitive visual perspective which improve how a user view and understand the displayed data. This simply combining prior art elements according to known methods to yield predictable results; use of known technique to improve similar devices (methods, or products) in the same way; and applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP 2143).
Conclusion
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure.
US Patent Application Publication No. 20220101438 filed by Gao et al. that disclose training Variational Autoencoder to map historical changes to latent space variables See at least ¶172
Examiner has pointed out particular references contained in the prior arts of record in the body of this action for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and Figures may apply as well. It is respectfully requested from the applicant, in preparing the response, to consider fully the entire references as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior arts or disclosed by the examiner. It is noted that any citation to specific pages, columns, figures, or lines in the prior art references any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331-33, 216 USPQ 1038-39 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED ABOU EL SEOUD whose telephone number is (303)297-4285. The examiner can normally be reached Monday-Thursday 9:00am-6:00pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMED ABOU EL SEOUD/Primary Examiner, Art Unit 2148
Prosecution Insights