DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 02/03/26 have been fully considered but they are not persuasive.
Applicant argues that the prior art of record does not teach receiving, by an intelligent router of the switchboard platform, a first request from an application, the first request comprising at least a portion of a prompt and a set of policy parameters (Amendment, pages 8, 9).
The examiner disagrees, since Gunmashta et al. disclose “The routing service 103 can authenticate any request from any tenant, and then route the request for service by ML models to any serving container 115 in a cluster of serving containers… The ML serving infrastructure 100 receives requests from tenants via a machine-learning service (MLS) gateway 101 or a similar interface. The MLS gateway 101 or similar interface receives a request from a tenant application and identifies a version or instance of an ML model associated with the request…The application metadata can specify the routing logic and ML framework version that is to be utilized to service the request. This information can be determined by performing a lookup of the application metadata from the data store 113 where the application metadata can be stored as an application tree and referenced by the application id of the requesting application.” (the tenants authentication information is considered as the policy parameters paragraphs 20 -22, 49).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gunmashta et al. (US PAP 2022/0382539) in view of De Wynter et al. (US PAP 2025/0117605).
As per claim 1, Gunmashta et al. teach a computer-implemented method for a switchboard platform to enable multiple disparate applications to query multiple disparate models (“The MLS gateway 101 or similar interface identifies model information associated with ML models corresponding to a cluster of available serving containers associated with the version of the ML model.”; paragraphs 13, 21), the method comprising:
receiving, by an intelligent router of the switchboard platform, a first request from an application, the first request comprising at least a portion of a prompt and a set of policy parameters (“The routing service 103 can authenticate any request from any tenant, and then route the request for service by ML models to any serving container 115 in a cluster of serving containers… The ML serving infrastructure 100 receives requests from tenants via a machine-learning service (MLS) gateway 101 or a similar interface. The MLS gateway 101 or similar interface receives a request from a tenant application and identifies a version or instance of an ML model associated with the request.”; paragraphs 20 – 22, 41 – 43, 49; the tenants identification represents the policy parameters);
selecting, by the intelligent router, a model of a sub-set of models at least partially based on at least one policy parameter in the set of policy parameters (“the application services 161 and/or MVS 163 can select an ML model and/or ML framework to utilize to service the prediction request (Block 311). The ML model and/or ML framework can be selected from the list of available ML models and/or ML frameworks that are enabled or active.”; paragraphs 22, 47, 49, 51);
determining, from a model registry of the switchboard platform, connection data for the model (“the model store 113 to get model identifiers (uniform resource identifiers (URIs))… The prediction services 115 can fetch a model URI from the model store 107 by model Id”; paragraphs 38, 47, 51);
transmitting, by the intelligent router and through a model connector of the switchboard platform, a second request for processing by the model, the second request being transmitted using the connection data and comprising at least a portion of the prompt (“the routing service 103 can split the incoming request into separate sub-requests, and then route the sub-requests to their corresponding clusters of serving containers… When requests are received by the routing service 103 via the MLS gateway 101, a check of the mapping managed by the routing manager 175 is made to determine if a requested ML model is executing using the service discovery and configuration system 111. If found, then the routing service 103 can forward the requests (or divide the request into a set of sub-requests) to the identified serving containers 115.”; paragraphs 21, 26, 33);
receiving, by the intelligent router, a response from the model; and
transmitting the response to the application (“If the prediction services returns a result for the prediction request, then the prediction result can be returned to the requesting tenant application”; paragraphs 21, 52).
However, Gumashta et al. do not specifically teach foundation models.
De WYNTER et al. disclose that Large language models (LLMs) are a type of foundation model which processes and generates natural language text. These models are trained on massive amounts of text data and learn to generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use foundation models as taught by De WYNTER et al. in Gumashta et al., because that would help generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
As per claim 8, Gumashta et al. teach a system, comprising:
one or more processors (“one or more processors”; paragraph 60); and
a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for a switchboard platform to enable multiple disparate applications to query multiple disparate models (“The MLS gateway 101 or similar interface identifies model information associated with ML models corresponding to a cluster of available serving containers associated with the version of the ML model…include a set of one or more processors coupled to one or more machine-readable storage media”; paragraphs 13, 21, 60), the operations comprising:
receiving, by an intelligent router of the switchboard platform, a first request from an application, the first request comprising at least a portion of a prompt and a set of policy parameters (“The routing service 103 can authenticate any request from any tenant, and then route the request for service by ML models to any serving container 115 in a cluster of serving containers… The ML serving infrastructure 100 receives requests from tenants via a machine-learning service (MLS) gateway 101 or a similar interface. The MLS gateway 101 or similar interface receives a request from a tenant application and identifies a version or instance of an ML model associated with the request.”; paragraphs 20 – 22, 41 – 43; the tenants identification represents the policy parameters);
selecting, by the intelligent router, a model of a sub-set of models at least partially based on at least one policy parameter in the set of policy parameters (“the application services 161 and/or MVS 163 can select an ML model and/or ML framework to utilize to service the prediction request (Block 311). The ML model and/or ML framework can be selected from the list of available ML models and/or ML frameworks that are enabled or active.”; paragraphs 22, 47, 49, 51);
determining, from a model registry of the switchboard platform, connection data for the model (“the model store 113 to get model identifiers (uniform resource identifiers (URIs))… The prediction services 115 can fetch a model URI from the model store 107 by model Id”; paragraphs 38, 47, 51);
transmitting, by the intelligent router and through a model connector of the switchboard platform, a second request for processing by the model, the second request being transmitted using the connection data and comprising at least a portion of the prompt (“the routing service 103 can split the incoming request into separate sub-requests, and then route the sub-requests to their corresponding clusters of serving containers… When requests are received by the routing service 103 via the MLS gateway 101, a check of the mapping managed by the routing manager 175 is made to determine if a requested ML model is executing using the service discovery and configuration system 111. If found, then the routing service 103 can forward the requests (or divide the request into a set of sub-requests) to the identified serving containers 115.”; paragraphs 21, 26, 33);
receiving, by the intelligent router, a response from the model; and
transmitting the response to the application (“If the prediction services returns a result for the prediction request, then the prediction result can be returned to the requesting tenant application”; paragraphs 21, 52).
However, Gumashta et al. do not specifically teach foundation models.
De WYNTER et al. disclose that Large language models (LLMs) are a type of foundation model which processes and generates natural language text. These models are trained on massive amounts of text data and learn to generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use foundation models as taught by De WYNTER et al. in Gumashta et al., because that would help generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
As per claim 15, Gumashta et al. teach a Computer-readable storage media coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for a switchboard platform to enable multiple disparate applications to query multiple disparate models (“The MLS gateway 101 or similar interface identifies model information associated with ML models corresponding to a cluster of available serving containers associated with the version of the ML model…include a set of one or more processors coupled to one or more machine-readable storage media”; paragraphs 13, 21, 60), the operations comprising:
receiving, by an intelligent router of the switchboard platform, a first request from an application, the first request comprising at least a portion of a prompt and a set of policy parameters (“The routing service 103 can authenticate any request from any tenant, and then route the request for service by ML models to any serving container 115 in a cluster of serving containers… The ML serving infrastructure 100 receives requests from tenants via a machine-learning service (MLS) gateway 101 or a similar interface. The MLS gateway 101 or similar interface receives a request from a tenant application and identifies a version or instance of an ML model associated with the request.”; paragraphs 20 – 22, 41 – 43; the tenants identification represents the policy parameters);
selecting, by the intelligent router, a model of a sub-set of models at least partially based on at least one policy parameter in the set of policy parameters (“the application services 161 and/or MVS 163 can select an ML model and/or ML framework to utilize to service the prediction request (Block 311). The ML model and/or ML framework can be selected from the list of available ML models and/or ML frameworks that are enabled or active.”; paragraphs 22, 47, 49, 51);
determining, from a model registry of the switchboard platform, connection data for the model (“the model store 113 to get model identifiers (uniform resource identifiers (URIs))… The prediction services 115 can fetch a model URI from the model store 107 by model Id”; paragraphs 38, 47, 51);
transmitting, by the intelligent router and through a model connector of the switchboard platform, a second request for processing by the model, the second request being transmitted using the connection data and comprising at least a portion of the prompt (“the routing service 103 can split the incoming request into separate sub-requests, and then route the sub-requests to their corresponding clusters of serving containers… When requests are received by the routing service 103 via the MLS gateway 101, a check of the mapping managed by the routing manager 175 is made to determine if a requested ML model is executing using the service discovery and configuration system 111. If found, then the routing service 103 can forward the requests (or divide the request into a set of sub-requests) to the identified serving containers 115.”; paragraphs 21, 26, 33);
receiving, by the intelligent router, a response from the model; and
transmitting the response to the application (“If the prediction services returns a result for the prediction request, then the prediction result can be returned to the requesting tenant application”; paragraphs 21, 52).
However, Gumashta et al. do not specifically teach foundation models.
De WYNTER et al. disclose that Large language models (LLMs) are a type of foundation model which processes and generates natural language text. These models are trained on massive amounts of text data and learn to generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use foundation models as taught by De WYNTER et al. in Gumashta et al., because that would help generate coherent and contextually relevant responses given a prompt or input text (paragraph 32).
As per claims 2, 9, and 16, Gumashta et al. in view of De WYNTER et al. further disclose the set of policy parameters comprises one or more of a tenant identifier, an application identifier, a domain, an intent, a task, and a modality (“An application identifier (application id) identifies a specific instance of an application type. Each application id allows each instance of an application type to have separately trained models associated therewith.”; Gumashta et al., paragraphs 41 - 43).
As per claims 3, 10, and 17, Gumashta et al. in view of De WYNTER et al. further disclose selecting, by the intelligent router, the foundation model of the sub-set of foundation models at least partially based on the at least one policy parameter in the set of policy parameters comprises applying a policy responsive to the first request, and the foundation model conforming to the policy (“The application metadata can specify the routing logic and ML framework version that is to be utilized to service the request. This information can be determined by performing a lookup of the application metadata from the data store 113 where the application metadata can be stored as an application tree and referenced by the application id of the requesting application.”; Gumashta et al., paragraphs 41 – 44, 49; De WYNTER et al., paragraphs 32 - 37).
As per claims 4, 11, and 18, Gumashta et al. in view of De WYNTER et al. further disclose during a pre- production phase: defining criteria; and selecting foundation models of a set of foundation models for inclusion in the sub-set of foundation models based on the criteria (“A data model information in the service discovery and configuration system 111 provides information about which serving containers 115 are expected to host-specific ML models (e.g., specific version) and which serving containers actually host the specified ML models…”; Gumashta et al., paragraphs 27- 33; De WYNTER et al., paragraphs 32 - 37).
As per claims 5, 12, and 19, Gumashta et al. in view of De WYNTER et al. further disclose selecting the foundation models of the set of foundation models for inclusion in the sub-set of foundation models based on the criteria comprises: transmitting requests to the foundation models in the set of foundation models in a first rank order based on cost to query; and determining a second rank order of foundation models that meet the criteria, at least a portion of the foundation models in the second rank order of foundation models being included in the sub-set of foundation models (“the clusters and serving containers operate other similar types of ML models other than scoring ML models such as ranking and recommendation models. Scoring is provided as an example rather than by limitation. The clusters can include in some implementations of ranking services and recommendation services, which support ranking models”; Gumashta et al., paragraphs 21 – 26; De WYNTER et al., paragraphs 32 - 37).
As per claims 6, 13, and 20, Gumashta et al. in view of De WYNTER et al. further disclose the connection data comprises an endpoint of the foundation model in a model serving infrastructure (“The processes of the ML serving infrastructure…The applications services 161 and/or MVS 163 can then determine the URI or similar identifier for the selected ML models and/or ML frameworks from the collected metadata”; Gumashta et al., paragraphs 44 - 51).
As per claims 7, 14, Gumashta et al. in view of De WYNTER et al. further disclose the set of foundation models comprises large language models (LLMs) [De WYNTER et al., paragraph 32].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEONARD SAINT-CYR whose telephone number is (571)272-4247. The examiner can normally be reached Monday- Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEONARD SAINT-CYR/Primary Examiner, Art Unit 2658
Prosecution Insights