DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on January 18, 2023. Claims 1-20 are pending in the case. Claims 1, 2, and 15 are the independent claims.
This action is non-final.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102€, (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-13 and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Arya et al. (US 20200349468 A1) in view of Thompson et al. (US 11003645 B2).
With respect to claim 1, Arya teaches a system for monitoring feature engineering workflows while labeling data for artificial intelligence model development (e.g. paragraph 0023, machine learning lifecycle including data annotation and feature engineering stages; paragraph 0025, supporting ML workloads with features for storage and management of data with respect to engineering teams and machine learning lifecycle; paragraph 0066, data definition and feature engineering in machine learning workflows), the system comprising: one or more preprocessors; and a non-transitory computer readable medium comprising instructions that when executed by the one or more preprocessors cause operations (e.g. paragraphs 0124-0125, ROM/storage devices storing instructions used by processing units to execute processes of described implementations) comprising:
receiving a first label for a first sample from a first dataset, wherein the first dataset is accessible to a first subset of a plurality of users (e.g. paragraph 0048, annotation object is a collection of labels describing entities in associated dataset; paragraph 0049, multiple different sets of annotation objects associated with same dataset; paragraph 0056, Fig. 4, illustrating example of annotation object associating labels corresponding to image identifiers; paragraph 0060, dataset shared among different projects/teams, and each project/team enabled to label and organize the data based on its own needs; paragraph 0061, using independent permissions and terms of use settings for datasets, annotations, and packages; paragraph 0066, creating labels; paragraph 0069, Fig. 7, code for creating a new version of an annotation object on a dataset object; paragraph 0072, creating new version of an annotation on dataset; i.e. a given user/team/project provides an annotation/label for a corresponding dataset object for use by a particular team/project);
receiving a first user input to generate first version metadata of the first label (e.g. paragraph 0038, source code including statements for data definition and feature engineering; user programs including code statements describing intent/type of request compiled into execution graphs for execution; paragraph 0041, users interacting with APIs for accessing data in ML metadata store; paragraph 0062, changes in data versioned just like software; paragraph 0066, publishing labels as new version of an annotation; paragraph 0069, Fig. 7, code for creating a new version of an annotation object on a dataset object; paragraph 0072, code for creating new version of annotation; clause creating revision version from a specified version and sub-clause specifying the version of the dataset which this annotation refers to; i.e. user-created source code or other API interaction, is provided as an input which indicates the manner in which versioning metadata is to be generated for a provided annotation/label);
in response to receiving the first user input, determining a feature engineering workflow for generating the first version metadata (e.g. paragraph 0041, ML metadata store including information corresponding to permissions, version information, and user information, including which user created respective object, last edited the object, auditing information, access information, etc.; paragraph 0060, each project/team able to label and organize data based on its own needs; paragraph 0066, data definition and feature engineering in machine learning workflows; creating dataset, using user supplied ML model to create labels and publish them as new version of annotation, creating split, package, training model; i.e. the first version metadata (i.e. version information associated with label/annotation) is associated with a particular feature engineering/machine learning workflow for a particular team/project and is information is known/determined in order to store the version of the annotation/label along with other associated metadata as belonging to the particular user/team/project workflow);
determining a first workspace corresponding to the feature engineering workflow (e.g. paragraph 0032, source code associated with machine learning model; paragraph 0038, source code including code to implement functionality associated with machine learning models, as well as for data definition and feature engineering; paragraph 0049, different annotation objects with corresponding sets of labels associated with different ML models for a same dataset; paragraph 0060, dataset serving as ground truth and shared among different projects/teams; each project/team able to label and organize data based on its own needs; paragraph 0062, versioning of data just like versioning of software; providing versioning scheme on all four high level objects; paragraph 0063, paragraph 0064, different ML projects able to share and evolve versions on their own cadence and needs without disrupting other projects, pin a specific version in order to reproduce training results, and track version dependencies between data and trained models; paragraphs 0063-0066, describing versioning of objects, including for feature engineering in machine learning workflows; i.e. the versioning metadata for the annotations/labels are stored along with the associated user/project/team/workflow to which it applies, and further along with dependencies upon other associated project information, including related machine learning models, where the combined machine learning model, dataset, annotations/labeling, versioning and dependencies for each of these, taken together with the particular team/project which they are associated with, are collectively analogous to a workspace corresponding to a feature engineering workflow);
determining, based on the first workspace, a first credential requirement for accessing the first version metadata (e.g. paragraph 0041, ML metadata store including information corresponding to permissions, version information, user information, etc.; paragraph 0043, performing authentication of users based on credentials etc. that request access to data stored in the system; determining whether user permitted to access data based on their level of access; determining whether access should be granted to objects; paragraph 0047, indicating that objects include annotation/label-type objects; i.e. credential information corresponding to access of data in the system, including version metadata and other associated user/team/project information (including data for the particular team/project and the associated workflow/workspace), is determined);
receiving, from a second user, a second user input requesting access to training data for an artificial intelligence model based on version metadata of labels in the first workspace (e.g. paragraph 0033, training machine learning models using generated datasets; paragraph 0043, users requesting access to data stored on system; determining whether access to data should be permitted; performing authentication of users; for users that are authenticated, different levels of access (viewer, consumer, owner, etc.) attributed to users requesting access to data; i.e. in addition to providing access to version metadata (annotation/label information including version information), access to the underlying dataset itself may also be requested and provided, such that a different user, belonging to a same or different team as the first user, may request access to the dataset/training data (which is possibly shared across multiple different projects/teams, each applying their own annotations/labels), where a second user belonging to a same project/team as the first user, or otherwise having credentials/permissions to access the versioning metadata for the annotations/labels of the first user (which may be associated with a particular team/project and workflow/workspace as discussed above), may request access based on those credentials/permissions);
in response to the second user input, determining whether to grant access to the training data based on a first comparison of user profile data for the second user and the first credential requirement (e.g. paragraph 0043, performing authentication of users; determining whether access to particular data should be permitted; authorization component determining whether users are permitted access based on level of access; determining whether access should be granted to datasets; paragraph 0044, at initial time dataset is requested, agreement to terms of use provided; upon agreement with terms of use, access to dataset granted; i.e. granting or not granting access to the underlying dataset based on the user’s data including credentials, level of access, etc.).
Although Arya generally describes a format of the dataset which may be provided/generated by the user (e.g. paragraphs 0051-0053, describing Fig. 3), Arya does not explicitly disclose:
wherein the feature engineering workflow comprises a plurality of feature nodes, and wherein each feature node corresponds to respective feature transformation data;
determining a current progress point of the feature engineering workflow;
determining a first feature node in the feature engineering workflow corresponding to the current progress point;
determining first feature transformation data corresponding to the first feature node;
determining, based on the first feature transformation data, a first workspace corresponding to the feature engineering workflow;
generating for display, in a user interface, the training data based on the first comparison.
However, Thompson teaches:
wherein the feature engineering workflow comprises a plurality of feature nodes, and wherein each feature node corresponds to respective feature transformation data (e.g. col. 14 lines 23-66, visual node graph conveying data dependencies and data transformations associated with selected columns; user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; col. 32 lines 19-51, visual node graph displaying representations of resources (such as datasets and columns/features within the datasets as shown in Figs. 3A-B) as nodes and dependency relationships between those resources; user drilling into any particular node and selecting any particular column of data within the represented data set; also includes information regarding data transformation applied to columns at each step of the pipeline);
determining a current progress point of the feature engineering workflow (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular transformation step/progress point, such as a current/currently-selected data transformation step in a data processing pipeline/feature engineering workflow);
determining a first feature node in the feature engineering workflow corresponding to the current progress point (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular feature node, such as a column or other data object, within the data transformation pipeline/feature engineering workflow, which corresponds to the current/currently-selected step/progress point);
determining first feature transformation data corresponding to the first feature node (e.g. col. 14 lines 31-66, user interface informing user of data transformations applied to selected column at each step of the data pipeline; column lineage metadata describing each transformation step of column through data lifecycle, including how the columns were transformed and the transformation code used at the transformation step; col. 16 lines 11-26, each data transformation step applying transformation code to source data sets to produce output data sets);
determining, based on the first feature transformation data, a first workspace corresponding to the feature engineering workflow (e.g. col. 14 lines 65-66, identifying transformation code used at transformation step; col. 16 lines 11-26, software code that defines instructions to transform source columns of source datasets into target columns of target datasets; col. 24, lines 12-34, storing versions of transformation code along with particular datasets, etc.; col. 25 lines 31-65, storing versioned transformation code, which may be in a programming language and include specifications of corresponding datasets, columns, and operations or functions to perform, each reference within the transformation code at each step of the data pipeline; col. 36 lines 19-37, describing user interface of Fig. 3 as including selectable tab 330 for displaying transformation code to the user; i.e. particular transformation data may be determined as corresponding to the data transformation pipeline/feature engineering workflow and corresponding workspace by the corresponding transformations being associated with a particular transformation step/progress point of the pipeline/workflow);
generating for display, in a user interface, the training data based on the first comparison (e.g. col. 31 lines 32-47, determining access control policies or permission/access levels and using information to inform user via user interface of the permissions/access levels granted to them for accessing columns and data sets, etc.; col. 47 line 17-col. 48 line 54, describing Fig. 8; user interface for allowing permissions associated with resources represented in visual graph node to be displayed; visual node graph displaying representations of resources, including data sets, with the nodes/resources/data sets displayed according to level of permissions granted to particular user for the resource represented by that node; multiple types of permissions associated with user for particular resource, such as data access in datasets, which refers to permissions associated with the actual data contained in a data set and whether a user can view those rows of data, or resource access which allows user to discover and see the existence of the resource/data set but not view the data contained inside it; i.e. based on determination/comparison of user’s information and access permissions/credentials, a user interface may be generated for displaying the datasets to the user, such as those datasets which user has permission to view).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to represent the feature engineering workflow as a plurality of feature nodes with respective transformation data, the capability to determine a current/currently-selected progress point of the feature engineering workflow/data transformation pipeline, determine a corresponding feature node, determine corresponding transformations, and determine a corresponding grouping of feature transformation source code files, and the capability to display, based on the comparison of user profile data and credential requirements for accessing a dataset, a GUI which includes the corresponding training data/dataset (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 2 and 15, Arya teaches a non-transitory computer readable medium comprising instructions that when executed by the one or more preprocessors cause operations (e.g. paragraphs 0124-0125, ROM/storage devices storing instructions used by processing units to execute processes of described implementations) comprising a method; and the method for monitoring feature engineering workflows while labeling data for artificial intelligence model development (e.g. paragraph 0023, machine learning lifecycle including data annotation and feature engineering stages; paragraph 0025, supporting ML workloads with features for storage and management of data with respect to engineering teams and machine learning lifecycle; paragraph 0066, data definition and feature engineering in machine learning workflows), the method comprising:
receiving a first label for a first sample from a first dataset, wherein the first dataset is accessible to a first subset of a plurality of users (e.g. paragraph 0048, annotation object is a collection of labels describing entities in associated dataset; paragraph 0049, multiple different sets of annotation objects associated with same dataset; paragraph 0056, Fig. 4, illustrating example of annotation object associating labels corresponding to image identifiers; paragraph 0060, dataset shared among different projects/teams, and each project/team enabled to label and organize the data based on its own needs; paragraph 0061, using independent permissions and terms of use settings for datasets, annotations, and packages; paragraph 0066, creating labels; paragraph 0069, Fig. 7, code for creating a new version of an annotation object on a dataset object; paragraph 0072, creating new version of an annotation on dataset; i.e. a given user/team/project provides an annotation/label for a corresponding dataset object for use by a particular team/project);
receiving a first user input to generate first version metadata of the first label (e.g. paragraph 0038, source code including statements for data definition and feature engineering; user programs including code statements describing intent/type of request compiled into execution graphs for execution; paragraph 0041, users interacting with APIs for accessing data in ML metadata store; paragraph 0062, changes in data versioned just like software; paragraph 0066, publishing labels as new version of an annotation; paragraph 0069, Fig. 7, code for creating a new version of an annotation object on a dataset object; paragraph 0072, code for creating new version of annotation; clause creating revision version from a specified version and sub-clause specifying the version of the dataset which this annotation refers to; i.e. user-created source code or other API interaction, is provided as an input which indicates the manner in which versioning metadata is to be generated for a provided annotation/label);
in response to receiving the first user input, determining a feature engineering workflow for generating the first version metadata (e.g. paragraph 0041, ML metadata store including information corresponding to permissions, version information, and user information, including which user created respective object, last edited the object, auditing information, access information, etc.; paragraph 0060, each project/team able to label and organize data based on its own needs; paragraph 0066, data definition and feature engineering in machine learning workflows; creating dataset, using user supplied ML model to create labels and publish them as new version of annotation, creating split, package, training model; i.e. the first version metadata (i.e. version information associated with label/annotation) is associated with a particular feature engineering/machine learning workflow for a particular team/project and is information is known/determined in order to store the version of the annotation/label along with other associated metadata as belonging to the particular user/team/project workflow);
determining, based on the feature engineering workflow, a first grouping of source code files corresponding to the feature engineering workflow (e.g. paragraph 0032, source code associated with machine learning model; paragraph 0038, source code including code to implement functionality associated with machine learning models, as well as for data definition and feature engineering; paragraph 0049, different annotation objects with corresponding sets of labels associated with different ML models for a same dataset; paragraph 0060, dataset serving as ground truth and shared among different projects/teams; each project/team able to label and organize data based on its own needs; paragraph 0062, versioning of data just like versioning of software; providing versioning scheme on all four high level objects; paragraph 0063, paragraph 0064, different ML projects able to share and evolve versions on their own cadence and needs without disrupting other projects, pin a specific version in order to reproduce training results, and track version dependencies between data and trained models; paragraphs 0063-0066, describing versioning of objects, including for feature engineering in machine learning workflows; i.e. the versioning metadata for the annotations/labels are stored along with the associated user/project/team/workflow to which it applies, and further along with dependencies upon other associated project information, including related machine learning models, where the machine learning model and the associated versioning may include a group of source code files (such as source code for the machine learning model associated with the versioned annotations/labels and also source code used to generate the versioned annotations/labels as shown in Fig. 7));
determining, based on the first grouping of source code files, a first credential requirement for accessing the first version metadata (e.g. paragraph 0041, ML metadata store including information corresponding to permissions, version information, user information, etc.; paragraph 0043, performing authentication of users based on credentials etc. that request access to data stored in the system; determining whether user permitted to access data based on their level of access; determining whether access should be granted to objects; paragraph 0047, indicating that objects include annotation/label-type objects; i.e. credential information corresponding to access of data in the system, including version metadata and other associated user/team/project information (including associated source code files as previously discussed), is determined);
receiving, from a second user, a second user input requesting access to training data for an artificial intelligence model based on version metadata of labels in the first grouping of source code files (e.g. paragraph 0033, training machine learning models using generated datasets; paragraph 0043, users requesting access to data stored on system; determining whether access to data should be permitted; performing authentication of users; for users that are authenticated, different levels of access (viewer, consumer, owner, etc.) attributed to users requesting access to data; i.e. in addition to providing access to version metadata (annotation/label information including version information), access to the underlying dataset itself may also be requested and provided, such that a different user, belonging to a same or different team as the first user, may request access to the dataset/training data (which is possibly shared across multiple different projects/teams, each applying their own annotations/labels), where a second user belonging to a same project/team as the first user, or otherwise having credentials/permissions to access the versioning metadata for the annotations/labels of the first user (which may be indicated in at least one source code file as discussed above and shown in Fig. 7), may request access based on those credentials/permissions);
in response to the second user input, determining whether to grant access to the training data based on a first comparison of user profile data for the second user and the first credential requirement (e.g. paragraph 0043, performing authentication of users; determining whether access to particular data should be permitted; authorization component determining whether users are permitted access based on level of access; determining whether access should be granted to datasets; paragraph 0044, at initial time dataset is requested, agreement to terms of use provided; upon agreement with terms of use, access to dataset granted; i.e. granting or not granting access to the underlying dataset based on the user’s data including credentials, level of access, etc.).
Although Arya generally describes a format of the dataset which may be provided/generated by the user (e.g. paragraphs 0051-0053, describing Fig. 3), Arya does not explicitly disclose generating for display, in a user interface, the training data based on the first comparison. However, Thompson teaches generating for display, in a user interface, the training data based on the first comparison (e.g. col. 31 lines 32-47, determining access control policies or permission/access levels and using information to inform user via user interface of the permissions/access levels granted to them for accessing columns and data sets, etc.; col. 47 line 17-col. 48 line 54, describing Fig. 8; user interface for allowing permissions associated with resources represented in visual graph node to be displayed; visual node graph displaying representations of resources, including data sets, with the nodes/resources/data sets displayed according to level of permissions granted to particular user for the resource represented by that node; multiple types of permissions associated with user for particular resource, such as data access in datasets, which refers to permissions associated with the actual data contained in a data set and whether a user can view those rows of data, or resource access which allows user to discover and see the existence of the resource/data set but not view the data contained inside it; i.e. based on determination/comparison of user’s information and access permissions/credentials, a user interface may be generated for displaying the datasets to the user, such as those datasets which user has permission to view).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to display, based on the comparison of user profile data and credential requirements for accessing a dataset, a GUI which includes the corresponding training data/dataset (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 3 and 16, Arya in view of Thompson teaches all of the limitations of claims 2 and 15 as previously discussed, and Arya and Thompson both further teach determining, based on the first grouping of source code files, the first credential requirement for accessing the first version metadata further comprises:
determining a first task of the feature engineering workflow (e.g. Arya paragraph 0027, owner regenerating labels or annotations; paragraph 0043, attributing different levels of access to users such as viewer, consumer, owner, etc.; i.e. the system may determine different types of tasks to be performed with respect to the data such as viewing, consuming, and/or owning (and tasks associated with data ownership such as generating labels or annotations, etc.); Thompson col. 30 lines 34-65, determining particular actions users are permitted to take with respect to particular columns, etc., i.e. the system may determine the various different types of action which users may take in the system with respect to resources/datasets); and
adjusting, based on the first task, the first credential requirement for accessing the first version metadata (e.g. Arya paragraph 0043, attributing different levels of access to users such as viewer, consumer, owner, etc.; i.e. the system may assign/adjust the user’s credentials/access permissions/levels based on determining that the user is to be associated with the corresponding task/role; Thompson col. 30 lines 34-65, associating users with defined permissions/access levels that specify actions the user is granted by default; allowing user’s access permissions/levels to be edited, such as modifying user’s permissions/access levels to enable certain kind of access (and corresponding actions/tasks)).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to determine various types of actions/tasks that may be performed with respect to the datasets/resources (including corresponding labels and version metadata as taught by Arya), and adjust credential requirements for accessing the datasets/resources based on these actions/tasks (as taught by Thompson, such as by associating particular actions/tasks with particular access levels/permissions, and/or further associating, or modifying associations of, those actions/tasks with a particular user via granted access levels/permissions). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 4 and 17, Arya in view of Thompson teaches all of the limitations of claims 3 and 16 as previously discussed, and Arya and Thompson both further teach wherein determining, based on the first grouping of source code files, the first credential requirement for accessing the first version metadata further comprises:
determining a first user for performing the first task (e.g. Arya paragraph 0027, finding errors in dataset, identifying dependent and derived data, as well as corresponding owners; paragraph 0043, attributing different levels of access to users such as viewer, consumer, owner, etc.; i.e. the system may identify the particular user as being associated with the particular task/role; Thompson col. 30 lines 34-65, noting that particular users are associated with particular actions/tasks based on their default and specifically granted permissions/access levels); and
generating a notification to the first user (e.g. Arya paragraph 0027, notifying owners to regenerate labels or annotations; Thomson col. 31 lines 32-47, determining access control policies or permission/access levels and using information to inform user via user interface of the permissions/access levels granted to them for accessing columns and data sets, etc.; col. 47 line 17-col. 48 line 54, describing Fig. 8; user interface for allowing permissions associated with resources represented in visual graph node to be displayed; visual node graph displaying representations of resources, including data sets, with the nodes/resources/data sets displayed according to level of permissions granted to particular user for the resource represented by that node; multiple types of permissions associated with user for particular resource, such as data access in datasets, which refers to permissions associated with the actual data contained in a data set and whether a user can view those rows of data, or resource access which allows user to discover and see the existence of the resource/data set but not view the data contained inside it; i.e. the user may be notified, via user interface, regarding the particular permissions/access levels (and therefore corresponding actions/tasks) which the user has been granted).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to determine that a user is associated with a particular action/task (based on access levels/permissions) and display, based on the comparison of user profile data and credential requirements for accessing a dataset, a GUI which includes the corresponding training data/dataset (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 5 and 18, Arya in view of Thompson teaches all of the limitations of claims 2 and 15 as previously discussed, and Thompson further teaches wherein determining, based on the feature engineering workflow, the first grouping of source code files corresponding to the feature engineering workflow further comprises:
determining a current progress point of the feature engineering workflow (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular transformation step/progress point, such as a current/currently-selected data transformation step in a data processing pipeline/feature engineering workflow);
determining a first feature node in the feature engineering workflow corresponding to the current progress point (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular feature node, such as a column or other data object, within the data transformation pipeline/feature engineering workflow, which corresponds to the current/currently-selected step/progress point);
determining first feature transformation data corresponding to the first feature node (e.g. col. 14 lines 31-66, user interface informing user of data transformations applied to selected column at each step of the data pipeline; column lineage metadata describing each transformation step of column through data lifecycle, including how the columns were transformed and the transformation code used at the transformation step; col. 16 lines 11-26, each data transformation step applying transformation code to source data sets to produce output data sets); and
determining, based on the first feature transformation data, that the first grouping of source code files corresponds to the feature engineering workflow (e.g. col. 14 lines 65-66, identifying transformation code used at transformation step; col. 16 lines 11-26, software code that defines instructions to transform source columns of source datasets into target columns of target datasets; col. 24, lines 12-34, storing versions of transformation code along with particular datasets, etc.; col. 25 lines 31-65, storing versioned transformation code, which may be in a programming language and include specifications of corresponding datasets, columns, and operations or functions to perform, each reference within the transformation code at each step of the data pipeline; col. 36 lines 19-37, describing user interface of Fig. 3 as including selectable tab 330 for displaying transformation code to the user; i.e. particular source code files may be determined as corresponding to the data transformation pipeline/feature engineering workflow by the corresponding transformations being associated with a particular transformation step/progress point of the pipeline/workflow).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to determine a current/currently-selected progress point of the feature engineering workflow/data transformation pipeline, determine a corresponding feature node, determine corresponding transformations, and determine a corresponding grouping of feature transformation source code files (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 6 and 19, Arya in view of Thompson teaches all of the limitations of claims 2 and 15 as previously discussed, and Arya further teaches wherein determining, based on the feature engineering workflow, the first grouping of source code files corresponding to the feature engineering workflow further comprises:
determining that the first user has access to the first grouping of source code files (e.g. paragraph 0032, source code associated with machine learning model; paragraph 0038, source code including code to implement functionality associated with machine learning models, as well as for data definition and feature engineering; paragraph 0041, ML metadata store including information corresponding to permissions, version information, user information, etc.; paragraph 0043, performing authentication of users based on credentials etc. that request access to data stored in the system; determining whether user permitted to access data based on their level of access; determining whether access should be granted to objects; paragraph 0047, indicating that objects include annotation/label-type objects; i.e. where the particular user has corresponding access levels/permissions for different data objects in the system, including corresponding source code files for a given ML model, etc.); and
determining, based on the first user having access to the first grouping of source code files, that the first grouping of source code files corresponds to the feature engineering workflow (e.g. paragraph 0049, different annotation objects with corresponding sets of labels associated with different ML models for a same dataset; paragraph 0060, dataset serving as ground truth and shared among different projects/teams; each project/team able to label and organize data based on its own needs; paragraph 0062, versioning of data just like versioning of software; providing versioning scheme on all four high level objects; paragraph 0063, paragraph 0064, different ML projects able to share and evolve versions on their own cadence and needs without disrupting other projects, pin a specific version in order to reproduce training results, and track version dependencies between data and trained models; paragraphs 0063-0066, describing versioning of objects, including for feature engineering in machine learning workflows; i.e. the versioning metadata for the annotations/labels are stored along with the associated user/project/team/workflow to which it applies, and further along with dependencies upon other associated project information, including related machine learning models, where the machine learning model and the associated versioning may include a group of source code files (such as source code for the machine learning model associated with the versioned annotations/labels and also source code used to generate the versioned annotations/labels as shown in Fig. 7), such that the source code files are associated with the user having access to them and therefore to the corresponding workflow that the user is associated with).
Thompson further teaches wherein determining, based on the feature engineering workflow, the first grouping of source code files corresponding to the feature engineering workflow further comprises:
determining a current progress point of the feature engineering workflow (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular transformation step/progress point, such as a current/currently-selected data transformation step in a data processing pipeline/feature engineering workflow);
determining a first feature node in the feature engineering workflow corresponding to the current progress point (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 16, lines 11-26, describing data processing pipeline as including an ordered set of logic performing multi-step transformation of data obtained from data sources to produce output data sets, including a plurality of transformation steps; i.e. determining a particular feature node, such as a column or other data object, within the data transformation pipeline/feature engineering workflow, which corresponds to the current/currently-selected step/progress point);
determining a first user responsible for performing tasks corresponding to the first feature node (e.g. col. 30 lines 34-65, noting that particular users are associated with particular actions/tasks based on their default and specifically granted permissions/access levels and also including column level access control policy, etc.);
determining that the first user has access to the first grouping of source code files (e.g. col. 30 lines 34-65, generally defined permissions/access levels specify actions user is granted by default and may be applied against column level access control policy of each column in order to determine the access level and corresponding actions the user is permitted to take with regards to that particular column (such as to view, edit, etc. the selected feature/column and therefore the corresponding transformation code which is provided as part of viewing the column as cited above)).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to determine a current/currently-selected progress point of the feature engineering workflow/data transformation pipeline, determine a corresponding feature node, a particular user able/responsible for performing actions/tasks with respect to the feature/column, and determine that the user has access to the corresponding source code files (as taught by Thompson, such as transformation code files associated with a data transformation for a particular column which the user has associated actions/tasks/permissions). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claims 7 and 20, Arya in view of Thompson teaches all of the limitations of claims 2 and 15 as previously discussed, and Arya further teaches wherein generating the training data further comprises generating for display native data corresponding to the first sample (e.g. paragraph 0052, Fig. 3, user utilizing object management API and data layer API to generate dataset object 300 represented in tabular format as a table with a separate row for each file; each row includes a column for an image identifier, a filename, and a thumbnail representation of an image corresponding to the filename; paragraph 0054, datasets for machine learning containing list of raw files; i.e. user-generated dataset information is generated in a format visible to the user, including raw dataset information; compare with paragraph 036 of the specification of the instant application, indicating that native data comprises data formats comprising data that originates from or relates to unlabeled data sourced from a raw data source).
Thompson further teaches wherein generating the training data further comprises:
generating for display data corresponding to the first sample (e.g. col. 14 lines 31-66, user able to use user interface to drill into any particular node of visual node graph and select any particular columns of data; column lineage metadata describing each data transformation step; col. 35 line 53-col. 36 line 18, Fig. 3B, selecting representation in visual node graph resulting in information being presented about the selection such as in pane 320 displaying information associated with selected column in dataset including details about the column such as name, description, dataset, date and time, typeclasses, statistics, dependencies, etc.), and
generating for display feature transformation data for the first sample (e.g. col. 14 lines 31-66, user interface informing user of data transformations applied to selected column at each step of the data pipeline; column lineage metadata describing each transformation step of column through data lifecycle, including how the columns were transformed and the transformation code used at the transformation step; col. 16 lines 11-26, each data transformation step applying transformation code to source data sets to produce output data sets; col. 36 lines 19-37, UI 300 includes selectable tab 330 to display code describing transformation or functions used to generate target column).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to for a user to select a given feature/sample, display information about the selected feature/sample (where this may include native data as taught by Arya), and further display corresponding transformation data/code (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claim 8, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Thompson further teaches wherein generating the training data further comprises: determining a first metric for samples in the first dataset; determining a second metric for samples in the training data; and generating for display a comparison of the first metric and the second metric (e.g. col. 45 line 31-col. 46 line 5, describing Fig. 6A-B, information panel 620 providing information associated with nodes in visual node graph such as breakdown of various categories for properties and attributes among resources represented by selected node or if no nodes are selected all nodes, including number of resources in selection for various categories including numbers of files associated with resources, size of files, frequent columns, etc.; user selecting, using cursor, six nodes for project 1 and panel 620 displays breakdown of properties and attributes of the resources represented by those six nodes; selection of category highlights corresponding nodes in that category, etc.; i.e. as shown in Figs. 6A-6B, the displayed nodes correspond to different datasets (analogous to at least a first dataset and training data/a training dataset), and metrics are determined for each of these datasets, and the corresponding metrics are displayed for comparison in the information panel, such that a user can compare the datasets, such as seeing datasets according to their corresponding file sizes, numbers of files, frequent columns, etc.).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya and Thompson in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models), to incorporate the teachings of Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions) to include the capability to determine metrics for the plurality of datasets (including at least a first dataset and a training dataset as taught by Arya) and display a comparison of the metrics in a user interface (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to provide improvements to user interfaces for resource/data set dependency tracking, including by showing representations of relationships between resources, projects, user permissions, etc. as described in Thompson (col. 3 lines 31-52).
With respect to claim 9, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Arya further teaches wherein generating the training data further comprises: receiving a user annotation to the training data; and storing the user annotation in the training data (e.g. paragraph 0033, training machine learning models using ML datasets; paragraph 0048, annotation object is a collection of labels describing entities in associated dataset; paragraph 0049, multiple different sets of annotation objects associated with same dataset; paragraph 0056, Fig. 4, illustrating example of annotation object associating labels corresponding to image identifiers; paragraph 0060, dataset shared among different projects/teams, and each project/team enabled to label and organize the data based on its own needs; paragraph 0061, using independent permissions and terms of use settings for datasets, annotations, and packages; paragraph 0066, creating labels; paragraph 0069, Fig. 7, code for creating a new version of an annotation object on a dataset object; paragraph 0072, creating new version of an annotation on dataset; i.e. a given user/team/project provides an annotation/label for a corresponding dataset object for use by a particular team/project).
With respect to claim 10, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Arya further teaches wherein generating the training data further comprises:
generating for display an option to annotate the first version metadata (e.g. paragraph 0041, indicating that ML metadata store includes relationships between objects (such as annotations) and users, including permissions, version information, and user information, including indications of users creating and editing objects; paragraphs 0072, code listing 750 to create a new version of an annotation on a dataset; creating revision version off of specified version to create new version of annotation, i.e. creating new version of annotation human_activity@1.3.0 off of specified version human_activity@1.2.0; i.e. a user may be able to edit/annotate various objects such as version information of annotations/labels, such as by editing corresponding source code files (as shown in Fig. 7, where this would be performed by the user via a corresponding interface) in which this version information/metadata is stored/designated);
receiving an annotation to the first version metadata (e.g. paragraphs 0041 and 0072, Fig. 7 as previously discussed, user can edit version information for labels/annotations, such as by editing corresponding source code, etc.); and
automatically modifying, based on the annotation, the version metadata of labels in the first grouping of source code files (e.g. paragraphs 0041 and 0072, Fig. 7 as previously discussed, user can edit version information for labels/annotations, such as by editing corresponding source code, etc.; paragraph 0049, ML model/application generating annotation object with labels for dataset; paragraph 0066, using user supplied ML model to create labels and publish them as new version of annotation; i.e. as cited, the source code listing for creating new versions of labels for a dataset (as shown in Fig. 7), which may be edited by a user, can be used to automatically modify version metadata of labels in associated files, such as by using an ML model/application to automatically generate the annotations/labels for the dataset (i.e. in accordance with versioning as provided in the corresponding code listing)).
With respect to claim 11, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Arya further teaches wherein receiving the first label for the first sample from the first dataset further comprises:
accessing a first table of the first dataset (e.g. paragraph 0041, permitting access to data stored in the system; paragraph 0052, Fig. 3, dataset object 300 represented in tabular format as a table with a separate row for each file; i.e. the dataset object may be stored in table form, and may be accessed for various purposes);
determining a relationship of the first table to a second table in the first dataset (e.g. paragraph 0053, primary key of dataset uniquely identifies entity in the dataset and in addition defines the foreign key in annotations to reference the associated entities in the datasets; paragraph 0055, Fig. 4, annotation object 400 associated with dataset object 300; i.e. an annotation object, also stored in table form, may be associated/determined to be associated/related with the dataset object in table form);
accessing a second table of the first dataset based on the relationship (e.g. paragraph 0041, permitting access to data stored in the system; paragraph 0056, Fig. 4, representation of annotation object 400 includes respective row for each label; information provided by annotation object 400 is derived from the dataset object, such that the annotation table is formed based on its relationship with the dataset table, and may be accessed for various purposes based on this relationship); and
retrieving the first label from the second table (e.g. paragraph 0041, permitting access to data stored in the system; paragraph 0056, Fig. 4, annotation object 400 includes respective labels that correspond to extracted features or supplementary properties of associated dataset object 300, where these respective labels can be accessed/retrieved from the annotation table for various purposes).
With respect to claim 12, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Arya further teaches wherein receiving the first label for the first sample from the first dataset further comprises:
querying a third datastore for the first sample, wherein the third datastore is accessible to a third subset of the plurality of users, and wherein the third datastore comprises unlabeled data sourced from the first dataset (e.g. paragraph 0041, user permissions for accessing objects; permitting access to data stored in the system; paragraph 0047, data model objects including datasets, annotations, splits, and packages; paragraph 0048, creating split object which is a collection of data subsets from an associated dataset; splitting dataset object into training set, testing set, validation set, etc.; paragraph 0057, user mixing an partitioning dataset in different ways; i.e. a given user/subset of users may have permissions to access/query a particular object, such as a split object with is a subset of unlabeled data based on/source from a parent dataset; i.e. a user or set of users having access to a training set, testing set, and/or validation set for a given ML model, where each of these are subsets based on an overall dataset, which may be unlabeled (i.e. unlabeled subset as shown in the split objects 500 and 510 of Fig. 5)); and
querying the first dataset for the first label, wherein the first dataset comprises a plurality of labeled data archives, wherein each of the plurality of labeled data archives is specific to a respective workspace (e.g. paragraph 0041, user permissions for accessing objects; permitting access to data stored in the system; paragraph 0052, Fig. 3, dataset object 300 represented in tabular format as a table with a separate row for each file; paragraph 0053, primary key of dataset uniquely identifies entity in the dataset and in addition defines the foreign key in annotations to reference the associated entities in the datasets; paragraph 0055, Fig. 4, annotation object 400 associated with dataset object 300; paragraph 0056, Fig. 4, representation of annotation object 400 includes respective row for each label; annotation object 400 includes respective labels that correspond to extracted features or supplementary properties of associated dataset object 300; paragraph 0060, each project/team labeling and organizing data based on its own needs and cadence; paragraphs 0062-0064, data versioned just like software, providing versioning for all four high-level objects (including dataset, annotation, split, and package), with version evolutions categorized into schema, revision, and patch; schema version signals that schema of data has changed; revision and patch denote that the data is updated, deleted, and/or new entities have been added; version management allowing different ML projects to share and evolve versions on their own cadence and needs without disrupting other projects, pin a specific version in order to reproduce training results, and track version dependencies between data and trained models; i.e. where the dataset object 300 and corresponding annotation object 400 (including multiple annotation objects having multiple versions of the labels), taken together, collectively comprise labeled data archives, each specific to a particular team/project (analogous to a workspace), where these respective labels can be accessed/queried from the annotation table via the dataset table for various purposes).
With respect to claim 13, Arya in view of Thompson teaches all of the limitations of claim 12 as previously discussed, and Arya further teaches wherein receiving the first label for the first sample from the first dataset further comprises:
determining a first labeled data archive corresponding to a current workspace (e.g. paragraph 0041, user permissions for accessing objects; permitting access to data stored in the system; paragraph 0052, Fig. 3, dataset object 300 represented in tabular format as a table with a separate row for each file; paragraph 0053, primary key of dataset uniquely identifies entity in the dataset and in addition defines the foreign key in annotations to reference the associated entities in the datasets; paragraph 0055, Fig. 4, annotation object 400 associated with dataset object 300; paragraph 0060, each project/team labeling and organizing data based on its own needs and cadence; paragraphs 0062-0064, data versioned just like software, providing versioning for all four high-level objects (including dataset, annotation, split, and package), with version evolutions categorized into schema, revision, and patch; schema version signals that schema of data has changed; revision and patch denote that the data is updated, deleted, and/or new entities have been added; version management allowing different ML projects to share and evolve versions on their own cadence and needs without disrupting other projects, pin a specific version in order to reproduce training results, and track version dependencies between data and trained models; i.e. for a particular project/team (i.e. having its own corresponding workspace/flow), attempting to access its particular corresponding dataset annotations/labels, a corresponding dataset object 300 (including a relevant version/revision) and annotation object 400 (including relevant version for the project/team) is determined); and
retrieving the first label from the first labeled data archive (e.g. paragraph 0041, user permissions for accessing objects; permitting access to data stored in the system; paragraph 0056, Fig. 4, representation of annotation object 400 includes respective row for each label; annotation object 400 includes respective labels that correspond to extracted features or supplementary properties of associated dataset object 300; i.e. where the retrieving the labels from the annotation object 400 version determined to be relevant to the requesting project/team/workspace via the dataset object 300 version/revision determined to be relevant to the requesting project/team/workspace).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Arya in view of Thompson, further in view of Trim et al. (US 20210064929 A1).
With respect to claim 14, Arya in view of Thompson teaches all of the limitations of claim 2 as previously discussed, and Arya further teaches retrieving the version metadata of labels from the training data (e.g. paragraph 0033, training machine learning models using ML datasets; paragraph 0041, indicating that ML metadata store includes relationships between objects (such as annotations/labels) and users, including permissions, version information, and user information, including indications of users creating and editing objects; users permitted to access/retrieve data objects (i.e. including version metadata of labels corresponding to training data); paragraph 0048, annotation object is a collection of labels describing entities in associated dataset; paragraph 0049, multiple different sets of annotation objects associated with same dataset; ). Arya and Thompson do not explicitly disclose
comparing the version metadata of labels for consistency; and
determining that the version metadata of labels have a threshold level of consistency.
However, Trim teaches
comparing the version metadata of labels for consistency (e.g. paragraphs 0037-0038, users/participants annotating dataset by labeling the dataset and submitting annotations; annotations are analyzed to determine whether they are in agreement; i.e. where a plurality of different sets of annotations for a dataset corresponding to a plurality of different users provides a plurality of different versions/version metadata of labels for the dataset corresponding to each user); and
determining that the version metadata of labels have a threshold level of consistency (e.g. paragraph 0038, determining whether annotations are in agreement; if in agreement and there is a consensus on the labeling, processed dataset is accepted; i.e. where determining that the different sets of labels/versions of labels corresponding to different users are in agreement is analogous to determining that the versions/version metadata has a threshold level of consistency).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Arya, Thompson, and Trim in front of him to have modified the teachings of Arya (directed to a data management platform for machine learning models) and Thompson (directed to column lineage for resource dependency system and graphical user interface, such as for managing datasets, their dependencies, and user access permissions), to incorporate the teachings of Trim (directed to detecting and preventing unwanted model training data, such as different sets of labels which are not in agreement) to include the capability to determine metrics for the plurality of datasets (including at least a first dataset and a training dataset as taught by Arya) and display a comparison of the metrics in a user interface (as taught by Thompson). One of ordinary skill would have been motivated to perform such a modification in order to prevent unwanted behavior in a model as described in Trim (paragraph 0003).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JEREMY L STANLEY/
Primary Examiner, Art Unit 2127
Prosecution Insights