Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Abstract Objection
The Abstract is objected to because the language does not comply with MPEP § 608.01(b). Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided. The abstract is objected because of the first eight words of the first sentence. See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 15 – 29 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The specific grounds are as follows:
Claim 15 -”the alert” lacks antecedent basis (claims 15-29).
Claim 15 recites “triggering an alert event” and then recites “wherein triggering the alert event comprises transmitting the alert from the platform to the central server.” There is insufficient antecedent basis for “the alert” in the claim. The only element previously introduced is “an alert event,” which is a different noun. The specification does not resolve whether the two terms identify the same thing. In places it uses them for the same step (“Triggering an alert in step 140 may comprise transmitting an alert” (para 0076)), while elsewhere it describes the alert as a distinct item having its own content (“The alert may include meta data, such as a timestamp, a rule identifier that triggered the alert and/or details to the performance data that triggered the alert.” (para 0081)) and describes alert events that are themselves about alerts (“an alert event indicating and alerting overload may be triggered” (para 0074)). It therefore cannot be determined whether “the alert” is the alert event itself or a separate message generated in response to the alert event. The claims themselves reflect this uncertainty: dependent claim 19 introduces “an alert” for the same platform-to-server transmission, which would be unnecessary if “the alert” had been introduced in claim 15. See MPEP § 2173.05(e); In re Packard, 751 F.3d at 1310, 1314. Claims 16-29 depend from, or incorporate all limitations of, claim 15 and are rejected for the same reason. For examination purposes, “the alert” is interpreted as any alert associated with the alert event. Applicant may overcome this rejection by, e.g., amending claim 15 to recite “wherein[AltContent: ] triggering the alert event comprises generating an alert and transmitting the alert from the platform to the central server.”
Exemplary language “in particular” (claims 17, 23, 24, 25 and 26).
Claim 17 recites “in particular by an alerting system comprised by the platform.” Claim 23 recites “in particular exceeds or falls below one or more of the determined thresholds;” Claim 24 recites “critical number of determined criteria above a certain,” followed by “in particular predetermined,” Claim 25 recites “in particular a second algorithm and/or AI model of the compute graph,” Claim 26 recites “in particular of a second algorithm and/or AI model deployed on the same or another platform.” In each of these claims, a broader recitation is followed by a narrower feature introduced by “in particular,” and the claim does not make clear whether the narrower feature is a required limitation. See MPEP § 2173.0S(d); Ex parte Wu, 10 USPQ2d 2031, 2033 (Bd. Pat. App. & Inter. 1989); Ex parte Steigerwald, 131 USPQ 74 (Bd. App. 1961); Ex parte Hall, 83 USPQ 38 (Bd. App. 1949). The specific uncertainties are as follows. In claim 17, it is unclear whether the triggering must be performed by an alerting system. In claim 23, it is unclear whether “meets” requires exceeding or falling below a threshold. In claim 24, it is unclear whether the priority level must be predetermined. In claim 25, it is unclear whether the monitored output data must be output of a second algorithm and/or AI model; the phrase is also grammatically ambiguous because it may modify either “output data” or “compute graph.” In claim 26, it is unclear whether the second algorithm and/or AI model must be deployed on a platform. The specification repeats the same wording (e.g., “in particular by an alerting system comprised by the platform” (para 0018)) and does not indicate which reading is intended. Applicant may overcome this rejection by deleting the “in particular” clauses or by positively reciting the narrower features, which may be presented in separate dependent claims.
Optional language “preferably” (claims 19, 23 and 24).
Claim 19 recites “preferably indicating at least one of a time and/or location at which the alert event was triggered,” which concerns the content of the alert. Claim 23 recites “preferably predetermined,” and claim 24 recites “preferably predetermined,” each concerning the “critical number” of criteria. These recitations leave unclear whether the indicated alert content (claim 19) and the advance determination of the critical number (claims 23 and 24) are required limitations of the claims. See MPEP § 2173.05(d).
Applicant may overcome this rejection by deleting “preferably” and either positively reciting or removing the feature that follows it.
Alternative articles “a, or the,” (claims 19 and 22); “stored alert” lacks antecedent basis (claim 22).
Claim 19 recites “transmitting a, or the, alert to an operator of the platform” and “storing a, or the, alert on the platform and/or the central server”. Claim 22 recites “applying an access restriction to a, or the, stored alert”. The construction “a, or the,” recites in the alternative both a newly introduced element and a previously introduced element, without identifying which previously introduced element is meant. Under the “the” alternative of claim 19 there are at least two candidate referents: the alert of claim 15 (which itself lacks antecedent basis, see ground (1)) and the alert introduced in claim 19 by “transmitting an alert from the platform to the central server,”.
The identity of the alert that must be transmitted to the operator or control system, or stored, therefore cannot be determined, and the claim is amenable to two or more plausible constructions. Ex parte Miyazaki, 89 USPQ2d 1207, 1211 (BPAI 2008) (precedential); MPEP §§ 2173.05(e), 2173.05(h). Claim 22 depends from claim 15, which recites neither a storing step nor a stored alert; storing is recited only in claim 19, which is not in the dependency chain of claim 22. The “the stored alert” alternative therefore lacks antecedent basis, and the claim does not identify which previously recited alert, if any, is “the stored alert.” Applicant may overcome this rejection by selecting a single article and, for claim 22, by identifying which alert is stored (e.g., by reciting storing the alert of claim 15, or by making claim 22 depend from a claim that recites storing).
Claim 19 - double inclusion of the transmission; “the trigger event” lacks antecedent basis.
Claim 15 already requires “wherein triggering the alert event comprises transmitting the alert from the platform to the central server.” Claim 19 again recites “transmitting an alert from the platform to the central server,” using the indefinite article. It is unclear whether claim 19 requires a second alert and a second transmission over the same path or refers back to the transmission of claim 15. This matters to scope because the alert content recited in claim 19 may attach either to the alert of claim 15 or to a further alert. See MPEP § 2173.05(o).
Claim 19 recites “and/or rules of the set of rules underlying the trigger event;”. There is insufficient antecedent basis for “the trigger event” in the claim. Neither claim 19 nor claim 15 introduces a trigger event; both recite an “alert event,” and claim 19 uses “the alert event” in the immediately preceding clause (“at which the alert event was triggered”). The specification does not define “trigger event” or its relationship to the alert event. It describes the trigger event as something that results from monitored data meeting a criterion (“monitored data resulting in a trigger event may be data that meets a criterion of the one or more criteria” (para 0015)), and it states that the trigger event may indicate that monitored data meets the thresholds (“The trigger event may indicate that one or more monitored data meets one or more of the determined thresholds” (para 0073)), without stating whether the trigger event is the claimed alert event, the occurrence of a criterion being met, or some other event. It therefore cannot be determined which rules are “underlying the trigger event.” See MPEP § 2173.05(e). Applicant may overcome these rejections by reciting, e.g., “wherein the alert transmitted to the central server indicates ...” and by replacing “the trigger event” with “the alert event.”
Claims 23 and 24 -”critical number.”
Claim 23 recites “critical number of determined criteria is met by the monitored data.” and claim 24 recites “critical number of determined criteria above a certain,”. In claims 23 and 24, “critical number” leaves unclear whether the indication merely reports that a selected count of criteria has been met, or additionally requires that the met criteria constitute a sufficiently high or sufficiently severe aggregate. The specification describes the latter concept only in relative terms (“it may be indicated that a relatively high number of thresholds are met and, thus, a relatively high number of alert events is triggered” (para 0074); “an (additional) alert event may be triggered if a certain number of particularly relevant, i.e. relatively high prioritised, thresholds are met” (para 0075)), and otherwise repeats the phrase (“a critical number of determined thresholds is met by the monitored data” (para 0073)). It provides no standard for measuring when a number becomes “critical” and does not indicate which meaning the claims require.
Because the claims leave this limitation unresolved, they fail to inform those skilled in the art about the scope of the invention with reasonable certainty. Nautilus, Inc. v. Biosig Instruments, Inc., 572 U.S. 898, 910 (2014); MPEP §§ 2173.02, 2173.05(b). The rejection is directed to this unresolved meaning, not to the absence of a specific numerical count. Applicant may overcome this rejection by reciting, e.g., “a predetermined threshold number of the determined criteria,”
Claim 25 -”the first AI model.”
Claim 25 recites “wherein the algorithm and/or AI model is a first algorithm and/or AI model and” and then “the compute graph comprising the first AI model;”. There is insufficient antecedent basis for “the first AI model” in the claim. Claims 15 and 25 present “algorithm” and “AI model” as alternatives joined by “and/or.” Where the first algorithm and/or AI model is an algorithm, the claim introduces no “first AI model,” and it is unclear whether the compute graph must contain the monitored first algorithm, must contain some AI model, or whether the algorithm alternative is excluded. The statement in the specification that the terms “AI model” and “(trainable) algorithm” may be used interchangeably (para 0057) does not resolve the question, because the claims themselves distinguish the two terms as alternatives, and the specification describes this embodiment using the full alternative form (“the output of connected (e.g. parent) nodes of the first algorithm and/or AI model within the compute graph” (para 0048)). See MPEP § 2173.05(e). Applicant may overcome this rejection by reciting “the compute graph comprising the first algorithm and/or AI model,11 or by expressly limiting the first algorithm and/or AI model to an AI model.
Claims 28 and 29 (and claim 27 in the alternative) - 35 U.S.C. 112(f) limitations without adequate corresponding structure.
Claims 28 and 29 recite “means adapted to carry out the method of claim 15” limitations of claims 28 and 29 invoke 35 U.S.C. 112(f). Among the functions those means must perform is the function recited in claim 15 as “based on a predetermined set of rules provided by the central server, determining one or more performance criteria, comprising one or more performance thresholds, for the algorithm and/or AI model”. For a computer-implemented function, the corresponding structure is a computer programmed with an algorithm that performs the claimed function, disclosed in sufficient detail (as a formula, flow chart, or prose sequence of steps) and clearly linked to that function. WMS Gaming, Inc. v. Int’l Game Tech., 184 F.3d 1339, 1349 (Fed. Cir. 1999); Aristocrat Techs. Australia Pty Ltd. v. Int’l Game Tech., 521 F.3d 1328, 1333 (Fed. Cir. 2008); MPEP § 2181, subsection II.B. Naming a component that performs the function, or restating the function, is not disclosure of an algorithm. Blackboard, Inc. v. Desire2Learn, Inc., 574 F.3d 1371, 1382-85 (Fed. Cir. 2009). Where an algorithm is disclosed for only part of the claimed function, the claim is indefinite. Noah Sys., Inc. v. Intuit Inc., 675 F.3d 1302, 1318-19 (Fed. Cir. 2012).
The specification restates the means claims without describing them (“there is provided a data processing platform comprising means for carrying out any of the above mentioned methods.” (para 0052); “the system comprising means for carrying out any of the above mentioned methods.” (para 0053)). The components it describes are identified only by name and function (“The first platform 320A comprises an alerting system 321A, a control system 322A, an operating system 323A and a storage system 324A.” (para 0077)). For the determining function, the specification restates the step (“Based on the provided set of rules, the performance thresholds are determined in step 240 (step 120 of method 100 shown in figure l).” (para 0067)), and the only structure it links to that step is an operator or controller acting through an operating system (“An operator or controller of the first platform 320A may determine and/or configure the thresholds via the operating system 313 of the central server 310.” (para 0078); “Based on the ontology, an operator of the first platform 320A may determine one or more thresholds via the operating system 323A.” (para 0079)). An operator is a person, and an operating system through which a person configures thresholds is a component, not a sequence of programmed steps by which the claimed means determines performance criteria based on the rules. The specification also states that the set of rules may include the thresholds (“The set of rules included in the first data 330A may comprise the thresholds determined in step 120 or step 240 as illustrated in figures land2.” (para 0078)), and that a rule may comprise identifiers, a relation keyword and a threshold value (para 0066). That disclosure describes thresholds that have already been determined in step 120 or step 240 and are then carried in the rules; it does not describe the acts by which the claimed means performs that determination in step 120 or step 240. In the same paragraph, the only agent identified as determining the thresholds is “An operator or controller of the first platform 320A” (para 0078), acting via an operating system. The transmission of the rules from control system 312 to control system 322A and the use of the thresholds in the comparison (paras. (0078), (0080) correspond to the separate receiving and comparing functions, not to the determining function. The specification therefore does not disclose an algorithm, clearly linked to the claimed means, for performing the entire determining function. Claims 28 and 29 are indefinite and are rejected under 35 U.S.C. 112(b). See MPEP § 2181, subsection II.B. In the alternative only, if claim 27 is construed under 35 U.S.C. 112(f) (see Claim Interpretation above), claim 27 is indefinite and is rejected under 35 U.S.C. 112(b) for the same reason.
Applicant may: (a) amend the claims so that the claim limitations will no longer be interpreted as limitations under 35 U.S.C. 112(f); (b) amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or (c) amend the written description of the specification such that it clearly links the structure, material, or acts disclosed in the specification to the entire claimed function, without introducing any new matter. If applicant is of the opinion that the written description already implicitly or inherently discloses the corresponding structure, applicant should clarify the record by stating on the record what that structure is and where it is disclosed. MPEP §§ 2181 and 2185.
Claim 29 - inconsistent disjunctive and conjunctive description of the execution environment.
Claim 29 recites “when executed by a data processing device or system,” and then “wherein the data processing device and the system comprise means adapted to carry out the method of claim 15.” The execution clause identifies “a data processing device or system,” while the wherein clause refers to “the data processing device and the system.” The claim does not make clear whether the means requirement applies to whichever execution entity is selected, or requires two distinct execution entities, each comprising the means, even where only one of them executes the instructions. The claim is therefore amenable to two or more plausible constructions. In re Packard, 751 F.3d at 1310, 1314; Ex parte Miyazaki, 89 USPQ2d at 1211; MPEP §§ 2173.05(e), 2173.05(h). Applicant may overcome this rejection by using consistent terminology for the execution entity and clarifying whether the means requirement applies to the selected execution entity or to two distinct execution entities.
Appropriate correction is required.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 – 20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or joint inventor, at the time the application was filed, had possession of the claimed invention.
When a claim containing a computer-implemented 35 U.S.C. 112(f) limitation is indefinite under 35 U.S.C. 112(b) for failure to disclose sufficient corresponding structure (an algorithm) in the specification, the claim also lacks adequate written description under 35 U.S.C. 112(a). MPEP § 2181, subsection IV; MPEP § 2161.01, subsection I; MPEP § 2163.03, subsection VI. As set forth in ground (8), the specification does not disclose an algorithm, clearly linked to the “means adapted to carry out the method of claim 15,” for determining the performance criteria based on the predetermined set of rules. The specification restates that function (para 0067) and links it only to an operator or controller acting through an operating system (paras. (0078), (0079), and its description of rule content (para 0066) does not describe how the claimed means performs the determining function. Accordingly, the specification does not describe the claimed means in sufficient detail that one of ordinary skill in the art would understand that the inventor possessed the claimed invention. Ariad Pharms., Inc. v. Eli Lilly & Co., 598 F.3d 1336, 1351 (Fed. Cir. 2010) (en banc); Vasudevan Software, Inc. v. MicroStrategy, Inc., 782 F.3d 671, 682-83 (Fed. Cir. 2015); MPEP § 2163. Claim 27 is rejected on this ground only under the alternative construction of claim 27 under 35 U.S.C. 112(f) set forth in the Claim Interpretation section.
Enablement
Claim 27 is rejected under 35 U.S.C. 112(a) as failing to comply with the enablement requirement. The claim is a single means claim of undue breadth.
Claim 27 recites “A data processing platform comprising means adapted to carry out the method of claim 15.” Its only element is a single means that is not recited in combination with any other element. The preamble “data processing platform” names the claimed article, as “Fourier transform processor” did in Hyatt, and the reference to claim 15 supplies the result to be achieved by that means. A single means claim covers every conceivable means for achieving the stated result, while the specification discloses at most only those means known to the inventor; such a claim is therefore not commensurate with the enabling disclosure. In re Hyatt, 708 F.2d 712, 714-15 (Fed. Cir. 1983); MPEP § 2164.08(a). The specification must enable the full scope of what is claimed. Amgen Inc. v. Sanofi, 598 U.S. 594, 610-13 (2023); MPEP § 2164.08.
Scope versus disclosure. Under its broadest reasonable interpretation, claim 27 covers any means whatsoever that carries out the method of claim 15.
The specification discloses one architecture, platform 320A comprising functionally identified systems (“The first platform 320A comprises an alerting system 321A, a control system 322A, an operating system 323A and a storage system 324A.” (para 0077); “The performance data of the AI model may be monitored by the control system 322A.” (para 0080); “Based on the comparison, an alert may be triggered by alerting system 321A (or, alternatively by alerting system 311).” (para 0081)). The claim is therefore broader than the enabling disclosure.
Consideration of the factors set forth in In re Wands, 858 F.2d 731, 737 (Fed. Cir. 1988) (MPEP § 2164.0l(a)):
Breadth of the claims: the claim is unrestricted, covering every means for performing the method of claim 15, whereas the specification discloses one platform architecture.
Nature of the invention: computer-implemented monitoring of algorithms and AI models deployed on edge devices with centrally provided rules; claim 27 recasts that method as an article defined solely by a single means for achieving its result.
State of the prior art: no reference is relied upon. The general availability of computing devices that execute software does not supply every means encompassed by an unrestricted single means claim.
Level of one of ordinary skill: a person of ordinary skill in software and distributed-systems implementation. Ordinary implementation skill does not resolve the unconfined breadth of a single means claim.
Level of predictability in the art: implementing the described platform architecture in software is predictable, but that predictability is confined to the described architecture and does not extend to the unrestricted scope of the single means.
Amount of direction provided by the inventor: limited to the platform architecture of paras. (0077)-(0081), whose components are identified by function; para 0052) merely restates the claim. No direction is given for any other means within the scope of the claim.
Existence of working examples: Figs. 1-3 and paras. (0056)-(0085) describe a single paper example at the block-diagram and flow-chart level; no example addresses means other than the described platform architecture.
Quantity of experimentation needed: the disclosure provides no teaching commensurate with the unrestricted single-means scope of claim 27; routine implementation of the described platform architecture does not enable the unrestricted means for the entire result, as addressed in Hyatt.
Weighing all of the Wands factors, and in particular the unrestricted breadth of the single means relative to the single disclosed architecture, the specification does not enable one of ordinary skill in the art to make and use the full scope of claim 27 without undue experimentation. Applicant may overcome this rejection by reciting a combination of structural elements, e.g., one or more processors and memory storing instructions that, when executed by the one or more processors, cause the platform to perform the recited steps.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.- Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 27, 28 and 29 are rejected under 35 U.S.C. 112(d) as being of improper dependent form for failing to include all the limitations of the claim upon which they depend.
Each of claims 27-29 refers to claim 15 (“A data processing platform comprising means adapted to carry out the method of claim 15.”; “the central server and the platform comprising means adapted to carry out the method of claim 15.”; “cause the data processing device or system to carry out the method of claim 15,”). Under MPEP § 608.0l(n), subsection III, a proper dependent claim must include every limitation of the claim from which it depends and specify a further limitation; a difference in statutory class alone does not establish improper dependency. Claim 15 is a method whose limitations are acts actually performed: monitoring, determining, comparing, triggering and transmitting. Claims 27-29, by contrast, define a platform, a system and a computer-readable medium that are merely capable of carrying out those acts. They do not require that any act of claim 15 be performed and therefore do not include every limitation of claim 15.
For example, the platform of claim 27 or the system of claim 28 may be made or sold, and the medium of claim 29 may be made or distributed, without any act of claim 15 being performed. A dependent claim that fails to include all limitations of the claim from which it depends is properly rejected under 35 U.S.C. 112(d). Pfizer, Inc. v. Ranbaxy Labs. Ltd., 457 F.3d 1284, 1291-92 (fed. Cir. 2006). Applicant may cancel the claims, amend the claims to place them in proper dependent form, rewrite the claims in independent form, or present a sufficient showing that the dependent claims comply with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 15, 17, 18, 23, 27, 28, 29 are rejected under 35 U.S.C. 103 as being unpatentable over Ren (US 2024/0129759 A1) in view of Song (US Pub. No. 2020/0160207 A1)
Per claim 15, Ren (US 2024/0129759 Al) is relied upon to teach a computer implemented method comprising: monitoring (reads on “The UE 115-c may monitor the status of an ML model used at the UE 115-c”, see Ren para 0112. Ren’s UE monitors the status of the ML model that is deployed and executed on the UE itself - “monitoring, by a platform,... performance data of ... [an AI model deployed on the platform”), by a platform, wherein the platform is an edge device (reads on “AUE 115 may include or may be referred to as a mobile device, a wireless device, a remote device, a handheld device, or a subscriber device, or some other suitable terminology”, see Ren para 0065. The UE is a mobile/remote end device at the edge of the radio access network -”the platform is an edge device”) of a computing system comprising a central server communicatively coupled to the platform (reads on “a second logical node, such as a centralized unit ML plane (CU-XP), which may operate as an ML model manager 310”, see Ren para 0097. The network side includes a centralized ML model manager that serves the UEs, i.e., a “computing system comprising a central server communicatively coupled to the platform”), performance data (reads on “the current confidence probability fails to satisfy a threshold value”, see Ren para 0116. The monitored quantity is the confidence probability produced by the ML model, which is model output data and therefore “performance data” as the spec anchor defines it (“a confidence of a prediction”, Spec para 0060).)of an algorithm and/or artificial intelligence, AI, model deployed on the platform (reads on “The UE 115-c may monitor the status of an ML model used at the UE 115-c”, see Ren para 0112. Ren’s UE monitors the status of the ML model that is deployed and executed on the UE itself - “monitoring, by a platform, ... performance data of ... [an AI model deployed on the platform”);[AltContent: ] based on a predetermined set of rules (reads on “The model status reporting configuration may indicate a method to detect the status of an ML model, a failure detection method, the content to include in a status report”, see Ren para 0104. The configuration is a set of rules (detection method, failure-detection method, report content) - “predetermined set of rules”) provided by the central server (reads on “For example, the UE 115-b may receive the ML model download from the model manager 310 (e.g., via the CU-CP 305), where the ML model download may include model information, a model status reporting configuration, or both”, see Ren para 0099. The status-reporting configuration is provided to the UE by the central model manager - “set of rules provided by the central server”), determining one or more performance criteria, comprising one or more performance thresholds (reads on “For example, the reporting configuration may define a threshold number of failure instances and a timer duration”, see Ren para 0115. A further performance threshold (threshold number of failure instances) is determined from the configured rules.), for the algorithm and/or AI model (reads on “Additionally or alternatively, a failure instance may be indicated by one or more pre-defined rules, such as if the current latent code distribution fails to satisfy an expected range, the current confidence probability fails to satisfy a threshold value, or any combination of these or other rules defining failure instances for one or more ML models”, see Ren para 0116. Ren expressly calls the rules “pre-defined” and each rule determines a performance criterion with a threshold (confidence-probability threshold value; expected range) - “determining one or more performance criteria, comprising one or more performance thresholds, for the ... AI model”), wherein the set of rules is associated with the algorithm, the AI model (reads on “Accordingly, the configuration of the reporting may be implicitly associated with a specific ML model (e.g., the ML model defined in the model download message)”, see Ren para 0105. The rule set is associated with a specific ML model - “associated with ... the AI model”) and/or the platform (reads on “Additionally, the UE 115-c may receive, from the base station 105-b, a configuration defining a trigger for reporting a status of the ML model”, see Ren para 0104. The configuration is delivered to, and defines a trigger for, the particular UE on which the model runs, so the rule set is also associated with the platform.); comparing, by the platform (reads on “In some other examples, the UE 115-c may analyze the status of an ML model and may transmit a status report for the ML model if one or more conditions of the ML model satisfy one or more threshold conditions (e.g., one or more pre-defined thresholds)”, see Ren para 0109. The UE itself determines whether the model’s monitored conditions satisfy pre-defined thresholds - “comparing, by the platform”), the monitored performance data to the one or more performance criteria (reads on “If the count value satisfies (e.g., equals or exceeds) the threshold number of failure instances before the timer expires, the UE 115-c may trigger a transmission of a report message”, see Ren para 0115. The UE tracks the failure count itself and tests it against the configured threshold number, which is a platform-side comparison of monitored data to a performance criterion); triggering an alert event (reads on “For example, the UE 115-c may be configured to actively transmit an ML model status report-or a request for resources to send an ML model status report-to the network in response to detecting a trigger for reporting the status of an ML model”, see Ren para 0111. Ren’s UE actively sends the status report in response to detecting the configured trigger, so the report is an event that the platform triggers in response to the rule-based determination) based on the comparison (reads on “In some cases, at 445 and in response to detecting the ML model failure, the UE 115-c may transmit a model failure indication to the base station 105-b”, see Ren para 0113. A model failure indication triggered by the threshold-based failure determination is an “alert event” “based on the comparison”) of the monitored data to the one or more performance criteria (reads on the configured threshold number, see Ren para 0115), wherein triggering the alert event comprises transmitting the alert from the platform to the central server (reads on “The CU-CP 305 may further forward reports (e.g., model status reports, model failure reports) from the UE 115-b to the model manager 310”, see Ren para 0101. Failure reports transmitted by the UE reach the central model manager - “transmitting the alert from the platform to the central server”). The prior art of record is silent on explicitly stating an edge device That monitors the performance of its own deployed ML model.
Song (US 2020/0160207 A1) is relied upon to teach central server communicatively coupled to the platform (reads on “transmitting information about the degraded performance of the machine learning model to a central server within the IoT network”, see Song para 0006. Song expressly names the coupled network node a “central server” in the same system as the edge device - “computing system comprising a central server communicatively coupled to the platform”); edge device... monitoring ... performance data of an ... AI model deployed on the platform (reads on “an edge device may monitor an accuracy of an ML model over time and initiate a model update with the cloud platform when the accuracy of the model falls below a predetermined threshold”, see Song para 0020. Song expressly teaches an “edge device” that monitors the performance (accuracy) of its own deployed ML model.); and transmitting the alert from the platform to the central server (reads on “transmitting information about the degraded performance of the machine learning model to a central server within the IoT network”, see Song para 0059. Song expressly transmits the degraded-performance notification from the edge device to the “central server”).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify Ren’s network-configured, UE-side ML-model status monitoring and failure reporting architecture by integrating the edge-device/central-server monitoring arrangement of Song, in which an edge device monitors its deployed ML model against rule-defined thresholds and reports degraded performance to a central server (see Song para 0044: “Pre-set thresholds or human defined thresholds for the metrics may be rules to determine the trigger of edge/cloud communication”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {A,G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have implemented Ren’s ML model manager 310 as a central server to which Ren’s edge UE transmits its threshold-triggered model failure report, as recited in the instant claims, by applying the threshold-rule/central-server reporting pattern of Song to Ren’s existing configuration-download and report-forwarding structure. As Song itself states: “Continual exchange or communication between edge and cloud may increase network cyber risk, cost of transmitting data, and burden to the network bandwidth. The example embodiments enable edge-cloud communication when it is necessary.” (see Song para 0045), which addresses the well-recognized problem of unnecessary edge-to-cloud traffic and network burden when edge-deployed models are monitored. One of ordinary skill in the art of distributed machine-learning deployment would have recognized that Ren’s logical model-manager node is, in a practical deployment, a centralized server serving many edge devices, and that Song’s edge-monitors/central-server-receives arrangement was a known, predictable solution to exactly that design point, requiring no change to Ren’s underlying UE-side monitoring, rule configuration, or reporting signaling. The combination is further supported by MPEP 2143(1) Rationale (A) (combining prior art elements according to known methods to yield predictable results) and Rationale (G) (some teaching, suggestion, or motivation in the prior art, here Song’s express statement of the bandwidth/cost benefit). The motivation to combine therefore arises from the references themselves; each element performs the same function in combination as it does separately, and one of ordinary skill would have had a reasonable expectation of success because both references implement threshold-based monitoring of a deployed ML model on an edge device with reporting to a network-side manager.
Per claim 17, the prior art of record further suggests the computer implemented method of claim 15, wherein the step of triggering the alert event is performed by the platform (reads on “In a second example, the model status reporting configuration may specify UE-triggered reporting 420 by the UE 115-c”, see Ren para 0111. Ren expressly labels this mode “UE-triggered reporting”: the edge device, not the network, decides to trigger the report, which is triggering by the platform.), in particular by an alerting system comprised by the platform (reads on “If the count value satisfies (e.g., equals or exceeds) the threshold number of failure instances before the timer expires, the UE 115-c may trigger a transmission of a report message”, see Ren para 0115. The UE logic that tracks the count and triggers the report transmission is a functional alerting component resident on the platform).
Per claim 18, the prior art of record further suggests the performance data comprises input (reads on “The model failure report may include input data (e.g., which may have caused the failure) to the ML model, output data from the ML model, statistics associated with the ML model”, see Ren para 0134. Ren’s failure report carries both the model’s input data and its output data, so the monitored/reported performance data comprises input and output data) and/or output data (reads on “Additionally or alternatively, a failure instance may be indicated by one or more pre-defined rules, such as if the current latent code distribution fails to satisfy an expected range, the current confidence probability fails to satisfy a threshold value, or any combination of these or other rules defining failure instances for one or more ML models”, see Ren para 0116. The confidence probability that Ren tests is produced by the ML model, i.e., it is output data of the model, matching the “output data” alternative and spec para 0060 (“a confidence of a prediction”)).
Per claim 23, the prior art of record further suggests triggering an alert event comprises: indicating that one or more of the monitored data meets one or more of the determined criteria (reads on “In some cases, at 445 and in response to detecting the ML model failure, the UE 115-c may transmit a model failure indication to the base station 105-b”, see Ren para 0113. The failure indication is sent precisely because the threshold criterion was met, so it indicates that the monitored data met the determined criterion), in particular exceeds or falls below one or more of the determined thresholds (reads on “For example, if using the ML model results in a communication reliability that fails to satisfy a threshold communication reliability, the UE 115-c may trigger transmission of a status report for the ML model in a periodic resource”, see Ren para 0109. The report indicates that the monitored data has fallen below the determined threshold - first prong).
Per claim 27, the prior art of record further suggests a data processing platform comprising means adapted to carry out the method of claim 15 (reads on “The instructions may be executable by the processor to cause the apparatus to receive, from a base station, ML model information defining an ML model for the UE, receive, from the base station, a configuration defining a trigger for reporting a status of the ML model, detect the trigger for reporting the status of the ML model based on the configuration, and transmit, to the base station, a report message indicating the status of the ML model based on detecting the trigger”, see Ren para 0007. Ren’s UE executes instructions that receive the rules, detect the rule-defined trigger and transmit the report, i.e., the same monitoring/comparing and triggering/transmitting functions performed by a programmed device; the algorithms are Ren para 0112 (monitoring), para 0115-0116 (criterion and comparison) and para 0101 (reporting), so the structure is at least equivalent to Applicant’s control and alerting systems).
Claim 28 is analyzed with respect to claim 15. The prior art of record further suggests a data processing system comprising a central server (reads on “transmitting information about the degraded performance of the machine learning model to a central server within the IoT network”, see Song para 0006. Song expressly names the network-side node a central server, which supplies the claimed central server) and a platform (reads on ‘The instructions may be executable by the processor to cause the apparatus to receive, from a base station, ML model information defining an ML model for the UE, receive, from the base station, a configuration defining a trigger for reporting a[AltContent: ] status of the ML model, detect the trigger for reporting the status of the ML model based on the configuration, and transmit, to the base station, a report message indicating the status of the ML model based on detecting the trigger”, see Ren para 0007. Ren’s UE executes instructions that receive the rules, detect the rule-defined trigger and report, i.e., the platform-side functions of claim 15, so its programmed structure is at least equivalent to Applicant’s control system 322A and alerting system 321A), the central server and the platform comprising means adapted to carry out the method of claim 15 (reads on “The instructions may be executable by the processor to cause the apparatus to transmit, to a UE, ML model information defining an ML model for the UE, transmit, to the UE, a configuration for the UE to report a status of the ML model, and receive, from the UE, a report message indicating the status of the ML model based on the configuration”, see Ren para 0022. Ren’s base-station apparatus executes instructions that deliver the rule configuration and receive the report, the server-side functions, by structure equivalent to central server 310 and control system 312).
Claim 29 is analyzed with respect to claim 15. The prior art of record further suggests a computer-readable medium comprising instructions (reads on “A non-transitory computer-readable medium storing code for wireless communications at a UE is described. The code may include instructions executable by a processor”, see Ren para 0009. Ren expressly discloses a non-transitory medium storing code executable by a processor to perform its UE method, which is the claimed medium and instructions) which, when executed by a data processing device or system, cause the data processing device or system to carry out the method of claim 15 (reads on “The instructions may be executable by the processor to cause the apparatus to receive, from a base station, ML model information defining an ML model for the UE, receive, from the base station, a configuration defining a trigger for reporting a status of the ML model, detect the trigger for reporting the status of the ML model based on the configuration, and transmit, to the base station, a report message indicating the status of the ML model based on detecting the trigger”, see Ren para 0007. The device executing Ren’s stored code is the UE apparatus of Ren para 0007, which performs the platform functions as analyzed for claim 27), wherein the data processing device and the system comprise means adapted to carry out the method of claim 15 (reads on “The instructions may be executable by the processor to cause the apparatus to transmit, to a UE, ML model information defining an ML model for the UE, transmit, to the UE, a configuration for the UE to report a status of the ML model, and receive, from the UE, a report message indicating the status of the ML model based on the configuration”, see Ren para 0022. The system side is Ren’s base-station apparatus of para 0022, which performs the server functions as analyzed for claim 28).
Claims 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song, and further in view of Brady (US 2020/0357259 Al).
Per claim 16, the prior art of record further teaches the steps of monitoring the data (reads on “The UE 115-c may monitor the status of an ML model used at the UE 115-c”, see Ren para 0112. The UE is the edge platform, and it is the UE itself that monitors the status of the ML model it executes; no network node performs this monitoring, which is the platform-side monitoring that spec para 0013 describes.), determining the one or more performance criteria (reads on “A failure instance for an ML model may be defined by the model failure detection method configuration (e.g., in the configuration message received at 410)”, see Ren para 0116. The UE applies the received failure-detection configuration to decide what counts as a failure instance, so the operative criteria are identified on the UE from the provided rules) and comparing the monitored data to the one or more performance criteria is performed by the platform (reads on “If the count value satisfies (e.g., equals or exceeds) the threshold number of failure instances before the timer expires, the UE 115-c may trigger a transmission of a report message”, see Ren para 0115. The comparison of the tracked failure count against the configured threshold is executed by the UE, so the comparing step is performed on[AltContent: ] the platform as claimed). The prior art of record uses a broad definition of determining. Brady is relied upon for a narrower definition of determining.
Brady (US 2020/0357259 A1) is relied upon to teach monitoring the data (reads on “Further, the system monitor 224 may compare multiple operational parameters to determine when to send an alert message, for example, varying an alert limit based on another parameter”, see Brady para 0031. Beyond reading a supplied value, Brady’s on-device system monitor itself varies (determines) the alert limit based on another parameter, which is determination of the criterion by the platform.), determining the one or more performance criteria (reads on “At block 504, an alert detection ruleset including alert limits is retrieved”, see Brady para 0044. The object the device retrieves is a ruleset that contains the alert limits, so identifying the applicable limit from the downloaded rules is done on the device; this matches the specification, under which the server-provided rules “may comprise the performance criteria” (Spec para 0014)) and comparing the monitored data to the one or more performance criteria is performed by the platform (reads on “At block 508, the results of the analysis are compared against the ruleset”, see Brady para 0048. Brady places the comparison of measured results against the ruleset on the monitored device itself, confirming on-device comparison).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song edge monitoring system by integrating Brady’s on-device rule evaluation, in which an edge device obtains a remotely provided ruleset, compares its measurements against the ruleset locally, and dispatches the resulting alert message to a control system as well as to a remote system (see Brady para 0044: “The ruleset may be retrieved from either a local on device source or via a remote source, e.g. a RESTful query, an HTTP GET, or other response requests, to a remote web service”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have performed the monitoring, threshold determination, and comparison on the edge platform itself and to have dispatched the alert to the platform’s control system in addition to transmitting it to the central server, as recited in the instant claims, by applying Brady’s local-ruleset evaluation and control-system message dispatch to Ren’s UE, which already receives its rule configuration from the network; Brady adds that an alert may include “a recommendation or an actuation to mitigate the breach” (see Brady para 0041), a reason to route the locally detected model failure to the platform’s control system as well. As Brady itself states: “Thus, instead of sending a continuous data stream to report a metric, an alert message may only be sent if the alert limit is breached, e.g., if a value is above or below the alert limit.” (see Brady para 0013), which addresses the well-recognized problem of power and bandwidth consumed by remotely-deployed devices that continuously report telemetry. One of ordinary skill in the art of edge/IoT monitoring would have recognized that Ren’s UE, a power- and bandwidth-constrained radio device, benefits from evaluating its rules locally and reporting only on breach, and that Brady’s technique was a known, predictable solution to exactly that need, requiring no change to Ren’s underlying model, configuration signaling, or report format. The combination is further supported by MPEP 2143(1) Rationale (C) (use of a known technique to improve similar devices in the same way) and Rationale (G) (Brady’s express motivation). The motivation to combine therefore arises from Brady itself, and one of ordinary skill would have had a reasonable expectation of success because dispatching a generated alert to a local control queue and local comparison against a downloaded rule set are routine software operations on the same processor-and-memory UE that Ren describes.
Claim 19 is analyzed with respect to claim 16. The prior art of record further suggests wherein triggering the alert event comprises: transmitting an alert from the platform to the central server (reads on “The CU-CP 305 may further forward reports (e.g., model status reports, model failure reports) from the UE 115-b to the model manager 310”, see Ren para 0101. Ren’s failure reports travel from the UE to the central model manager, which is the transmission to the central server recited in the first clause; the “preferably” content is optional), preferably indicating at least one of a time and/or location at which the alert event was triggered, and/or rules of the set of rules underlying the trigger event; transmitting a, or the, alert to an operator of the platform and (reads on “An alert message is not limited to an informational message that may inform an operator of the breach, but may also include a recommendation or an actuation to mitigate the breach”, see Brady para 0041. The alert is directed to an operator and to control actuation, matching the operator and control-system branches.)/or to a control system of the platform (reads on “For example, the message dispatcher 230 may place the alert message into a message queue for a control system”, see Brady para 0033. Brady’s dispatcher delivers the actual alert message to a control system, which satisfies the “to a control system of the platform” branch of the claim; in the combination Ren’s failure alert is also delivered to the platform’s control system by this known dispatch mechanism).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song, and further in view of Olgiati (US 2021/0097433 A1)
Per claim 20, the prior art of record does not explicitly state wherein the set of rules and/or the criteria are determined by an operator of the platform and/or (pre)determined based on one or more characteristics of the platform, the deployed algorithm and/or the AI model.
Olgiati (US 2021/0097433 Al) is relied upon to teach the set of rules and/or the criteria are determined by an operator of the platform (reads on “The thresholds and/or tiers may be specified by users or may represent defaults”, see Olgiati para 0038. Olgiati states that its alerting thresholds may be user-specified, which is a criterion determined by an operator) and/or (pre)determined based on one or more characteristics of the platform, the deployed algorithm and/or the AI model (reads on “Accordingly, the configuration of the reporting may be implicitly associated with a specific ML model (e.g., the ML model defined in the model download message)”, see Ren para 0105. Ren’s rule set is already model-specific; Olgiati supplies the express basis of the threshold on a model characteristic/operator input).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Olgiati’s teaching that the problem-detection threshold for a deployed ML model is set from a characteristic of that model (its training-data accuracy) or by a user (see Olgiati para 0052: “The threshold may be based (at least in part) on the accuracy of training data 112 or on a user-specified value”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(I) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have determined Ren’s model-specific failure-detection thresholds by an operator and/or based on a characteristic of the deployed ML model, as recited in the instant claims, by applying Olgiati’s threshold-setting practice to Ren’s model-specific reporting configuration. As Olgiati itself states, its approach is directed to “improving the quality of machine learning models in production by automatically detecting common problems and notifying users” (see Olgiati para 0017), which addresses the well-recognized problem of choosing alert thresholds that fit the particular model being monitored. One of ordinary skill in the art of ML-model monitoring would have recognized that a single fixed threshold would generate spurious or missed failure reports across models of differing baseline accuracy, and that Olgiati’s model-characteristic or user-specified threshold was a known, predictable solution to exactly that issue, requiring no change to Ren’s comparison or reporting steps. The combination is further supported by MPEP 2143(I) Rationale (C) (use of a known technique to improve similar methods in the same way) and Rationale (G). The motivation to combine therefore arises from Olgiati itself, and one of ordinary skill would have had a reasonable expectation of success because only the numerical value of the configured threshold changes.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song, and further in view of Lachos (WO 2021/188021 A1).
Per claim 21, the prior art of record does not explicitly state the set of rules comprises an ontology for defining and/or determining the performance criteria.
Lachos (WO 2021/188021 A1) is relied upon to teach the set of rules comprises an ontology (reads on “Based on the determined network elements and measurable metrics in relation to the network elements, an ontology schema is generated S130 identifying the measurable metrics in relation to the network elements, and conditions and arguments in relation to the measurable metrics in relation to the network elements”, see Lachos p. 8, ll.11-14. Lachos’s monitoring ontology schema identifies the measurable metrics and the conditions/arguments applied to them - an ontology for defining the monitoring criteria) for defining (reads on “For the related metrics conditions are related such as min, max, less than (LT), less than or equal (LE), equal (EQ), greater than or equal (GE), and greater than (GT)”, see Lachos p. 8, ll.17-19. The ontology carries relation keywords (LT, LE, EQ, GE, GT) - the same kind of “relation keywords” described in spec para 0035) and/or determining the performance criteria (reads on “The network service monitoring model has conditions in relation to numerical values of the measurable metrics in relation to the network elements”, see Lachos p. 8, l. 28 - p. 9, l. 1. Lachos’s ontology yields a monitoring model with concrete conditions on numerical metric values, which is the determination of criteria from the ontology).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Lachos’s monitoring ontology schema, which identifies measurable metrics together with relation conditions and arguments from which concrete monitoring conditions are generated (see Lachos p. 8, ll.17-19: “For the related metrics conditions are related such as min, max, less than (LT), less than or equal (LE), equal (EQ), greater than or equal (GE), and greater than (GT)”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {A, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill [AltContent: ]in the art to have expressed Ren’s network-provided model status reporting configuration as an ontology of monitorable quantities and relation keywords from which the failure-detection criteria are defined, as recited in the instant claims, by applying Lachos’s ontology-based monitoring-model generation to Ren’s network-configured rule set. As Lachos itself states: “Furthermore, instead of individually monitoring all metrics and elements that compose a service (e.g., virtual network functions, virtual links), the proposed solutions enable automated selection of metrics to be evaluated based on the requested network service and a monitoring ontology in relation to the network service to be monitored.” (see Lachos p. 7, ll.9-13), which addresses the well-recognized problem of manually specifying, metric by metric, what a network-deployed function should monitor. One of ordinary skill in the art of network-configured monitoring would have recognized that Ren’s configuration (detection method, failure-detection method, thresholds) is a structured description of metrics and conditions, and that Lachos’s ontology schema was a known, predictable way of expressing exactly such a description, requiring no change to Ren’s on-UE comparison and reporting. The combination is further supported by MPEP 2143(1) Rationale (A) (combining prior art elements according to known methods to yield predictable results) and Rationale (G). The motivation to combine therefore arises from Lachos itself, and one of ordinary skill would have had a reasonable expectation of success because both references concern network-node-generated monitoring conditions delivered to and applied by deployed network functions
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song, and further in view of Azure Alerts (“Overview of alerts in Microsoft Azure”, 2021)
Per claim 22, the prior art of record does not explicitly state applying an access restriction to a, or the, stored alert based on access restrictions to the platform, the algorithm and/or AI model and/or the monitored data.
Azure Alerts (“Overview of alerts in Microsoft Azure”, 2021) is relied upon to teach applying an access restriction to (reads on “The consumption and management of alert instances requires the user to have the Azure built-in roles of either monitoring contributor or monitoring reader”, see Azure Alerts p. 10, § “Azure role- based access control (Azure RBAC) for your alert instances”. Consumption and management of stored alert instances is gated on specific roles, which is an access restriction applied to the stored alert) a, or the, stored alert (reads on “All state changes are stored in the history of the alert”, see Azure Alerts p. 3, § “Manage alerts”. Azure stores each alert together with its history of state changes, so the alerts are “stored alerts” to which a restriction can be applied) based on access restrictions to the platform (reads on “For example, if a user only has monitoring contributor access for virtual machine ContosoVMl, that user can consume and manage only alerts generated on ContosoVMl”, see Azure Alerts p. 10, § “Azure role-based access control (Azure RBAC) for your alert instances”. The alert access restriction is derived from the user’s access rights to the monitored resource (the machine on which the alert was generated) - “based on access restrictions to the platform”), the algorithm and/or AI model and/or the monitored data.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating the alert-instance access control of Azure Alerts, in which stored alerts are viewable and manageable only by users holding access rights to the monitored resource on which the alert was generated (see Azure Alerts p. 10, § “Azure role-based access control (Azure RBAC) for your alert instances”: “For example, if a user only has monitoring contributor access for virtual machine ContosoVMl, that user can consume and manage only alerts generated on ContosoVM1”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {A, D}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have stored the model failure alerts received at Song’s central server and to have restricted access to each stored alert based on access rights to the platform from which it originated, as recited in the instant claims, by applying Azure’s resource-scoped alert access control to the alerts of Ren-Song. As Azure Alerts itself states: “Alerts proactively notify you when issues are found with your infrastructure or application using your monitoring data in Azure Monitor.” (see Azure Alerts p. 1, § “What are alerts in Microsoft Azure?”), which addresses the well-recognized need, in centralized monitoring of many monitored resources, to deliver alerts to the right users while confining each user to the alerts of resources that user is authorized for. One of ordinary skill in the art of centralized monitoring would have recognized that a central server collecting failure reports from many edge platforms holds information belonging to different platforms and operators, and that Azure’s alert role-based access control was a known, predictable solution to exactly that problem, requiring no change to the underlying detection and reporting. The combination is further supported by MPEP 2143(1) Rationale (A) and Rationale (D) (applying a known technique to a known system ready for improvement to yield predictable results). The motivation to combine therefore arises from the art itself, and one of ordinary skill would have had a reasonable expectation of success because role-based access control of stored records is a routine server-side function independent of how the alert was generated.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song, and further in view of Wenchel (US 2021/0174258)
Per claim 23, the prior art of record further suggests indicating that one or more of the monitored data meets (reads on “In some cases, at 445 and in response to detecting the ML model failure, the UE 115-c may transmit a model failure indication to the base station 105-b”, see Ren para 0113. The failure indication is sent precisely because the threshold criterion was met, so it indicates that the monitored data met the determined criterion) one or more of the determined criteria, in particular exceeds or falls below one or more of the determined thresholds (reads on “For example, if using the ML model results in a communication reliability that fails to satisfy a threshold communication reliability, the UE 115-c may trigger transmission of a status report for the ML model in a periodic resource”, see Ren para 0109. The report indicates that the monitored data has fallen below the determined threshold -first prong). The prior art of record is silent on explicitly stating indicating that a, preferably predetermined, critical number of determined criteria is met by the monitored data.
Wenchel (US 2021/0174258 Al) is relied upon to teach indicating that a, preferably predetermined, critical number of determined criteria is met by the monitored data (reads on “a compound alert can be: IF Individual Alert 1 fires AND Individual Alert 2 fires THEN ACT take model out of production”, see Wenchel para 0072. As an alternative mapping of the second permitted branch of claim 23, two individual criterion-alerts both firing is a predetermined critical number of two).
Before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Wenchel’s compound alert, which fires when a predetermined number of individual ML-monitoring criteria are each met (see Wenchel para 0072: “a compound alert can be: IF Individual Alert 1 fires AND Individual Alert 2 fires THEN ACT take model out of production”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(I) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have Ren’s alert indicate that a predetermined critical number of the configured criteria is met, as recited in the instant claims, by applying Wenchel’s compound-alert logic to Ren’s multiple configured failure rules (confidence threshold, expected range, throughput-loss threshold). As Wenchel itself states: “Alerts and their associated automated actions, semi-automated actions and/or human-initiated actions can take into account any of the metrics, scores, alerts, or any function of any subset of those elements” (see Wenchel para 0073), which addresses the well-recognized problem of acting on a single metric in isolation when monitoring a deployed ML model. One of ordinary skill in the art of[AltContent: ] ML monitoring would have recognized that Ren already defines several independent failure rules and that combining them into a count-based compound alert was a known, predictable technique, requiring no change to Ren’s individual rules. The combination is further supported by MPEP 2143(1) Rationale (C) and Rationale (G). The motivation to combine therefore arises from Wenchel itself, and one of ordinary skill would have had a reasonable expectation of success because the modification is simple Boolean logic over existing rule outcomes.
Claim 24 is alternately rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song and further in view of Gawlick (US 2011/0202490)
Per claim 24, the prior art of record does not explicitly state wherein the one or more determined criteria are each associated with a respective priority level, and wherein triggering an alert event comprises: indicating that a, preferably predetermined, critical number of determined criteria above a certain, in particular predetermined, priority level is met by the monitored data.
Gawlick (US 2011/0202490 Al) is relied upon to teach the one or more determined criteria are each associated with a respective priority level (reads on “The complex alert rule includes a plurality of alert rules, wherein each alert rule includes an indicator of a physiological characteristic of a patient associated with the alert rule and a condition priority associated with an urgency of the alert rule”, see Gawlick para 0006. Gawlick’s rules are defined by priority level, so each rule (criterion) carries a respective priority level), and wherein triggering an alert event comprises: indicating that a, preferably predetermined, critical number (reads on “Of course, the rule may also be based on more complex Boolean logic such as greater than or equal to some number of priority level alert rules being true”, see Gawlick para 0098. Gawlick frames the trigger as “greater than or equal to some number” of priority-level rules being true, a predetermined critical number; applied with the red-above-orange embodiment, it counts criteria above the selected boundary) of determined criteria above a certain, in particular predetermined, priority level (reads on “For example, a complex alert may be generated when one “red” or serious condition priority level alert rule is true, when three “orange” or less serious condition priority level alert rules are true, when five “black” or still less serious condition priority level alert rules are true, etc”, see Gawlick para 0098. In this embodiment “red” is the highest stated class, above the “orange” boundary, and the complex alert is generated when the critical number (one) of red criteria is true; the claim does not require the critical number to exceed one, and its open “comprises” language does not exclude the further lower-priority branches. This is an embodiment-based reading of “above a certain priority level”) is met by the monitored data (reads on “A complex alert is generated if the determined number of alert rules for a condition priority satisfies the trigger rule of the complex alert rule”, see Gawlick para 0006. Gawlick counts the satisfied criteria per priority level and generates an alert when that count satisfies the trigger rule, so the generated alert indicates that the critical number for the priority level is met).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Gawlick’s priority-level alert rules, which trigger when a specified number of alert rules of a selected priority level are true, including an embodiment that alerts on one rule of the highest (red) level (see Gawlick para 0098: “Of course, the rule may also be based on more complex Boolean logic such as Gawlick greater than or equal to some number of priority level alert rules being true”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(I) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have tagged Ren’s configured criteria with priority levels and to have Ren’s alert indicate that a critical number of criteria above a selected priority boundary is met, as exemplified by Gawlick’s alert when one “red” criterion is true, red being above the orange class, as recited in the instant claims, by applying that embodiment of Gawlick’s priority-count rule to Ren’s count-based failure trigger. As Gawlick itself states: “Ignoring these types of differences between patients may result in missed alarms, a high rate of false alarms, and alert fatigue, which may cause the medical staff to ignore alerts.” (see Gawlick para 0005), which addresses the well-recognized problem of false alarms and alert fatigue in any rule-based monitoring system. One of ordinary skill in the art of rule-based alerting would have recognized that this problem is not specific to the monitored subject (patients versus ML models) but to the alert logic itself, and that Gawlick’s priority-count rule was a known, predictable solution to it, requiring no change to Ren’s monitoring and reporting. The combination is further supported by MPEP 2143(1) Rationale (C) and Rationale (G). Gawlick is reasonably pertinent to the problem the inventor faced (urgency/severity classification of alerts from multiple thresholds, spec para 0075); see MPEP 2141.0l(a). One of ordinary skill would have had a reasonable expectation of success because the modification is Boolean logic over existing rule outcomes.
Claim 24 is alternately rejected under 35 U.S.C. 103 as being unpatentable over Ren (US 2024/0129759 A1) in view of and further in view of DataRobot (DataRobot, Inc., “Data Drift tab,” DataRobot MLOps documentation (page states “Updated November 5, 2021”).
Per claim 24, the prior art of record does not explicitly state wherein the one or more determined criteria are each associated with a respective priority level, and wherein triggering an alert event comprises: indicating that a, preferably predetermined, critical number of determined criteria above a certain, in particular predetermined, priority level is met by the monitored data.
DataRobot (DataRobot, Inc., “Data Drift tab,” DataRobot MLOps documentation (page states “Updated November 5, 2021”) is relied upon to teach the one or more determined criteria are each associated with (reads on “Define or override the list of high or low importance features”, see DataRobot p. 21, § “Customize data drift status”. Each per-feature drift criterion is tagged with an importance level (high or low) - “associated with a respective priority level”) a respective priority level (reads on “Configure the thresholds of the Importance metric”, see DataRobot p. 22, § “Customize data drift status”. The importance levels themselves are set by configurable thresholds, which confirms that each criterion’s priority level is a defined attribute of the criterion), and wherein triggering an alert event comprises (reads on “Configure the values that trigger drift statuses for At Risk”, see DataRobot p. 24, § “Customize data drift status”. DataRobot makes the status trigger values configurable, which is a predetermined critical number as claimed): indicating that a (reads on “you can configure the rule for a deployment to mark its drift status as At Risk if one of the following is true”, see DataRobot p. 24, § “Customize data drift status”. Marking the deployment’s drift status as At Risk is the claimed indicating step: it signals that the count rule was satisfied), preferably predetermined, critical number of determined criteria above a certain, in particular predetermined, priority level is met by the monitored data (reads on “the number of high importance features above the drift threshold is greater than 3”, see DataRobot p. 24, § “Customize data drift status”. DataRobot expressly triggers a status when the count of criteria met (features above drift threshold) among criteria of the high priority level exceeds a critical number (3) - the limitation verbatim in substance).
Before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating DataRobot’s prioritized drift-status rule, in which each monitored criterion (per-feature drift threshold) carries an importance level and the deployment status is set to Failing or At Risk when the number of high-importance criteria met exceeds a configured number (see DataRobot p. 24, § “Customize data drift status”: “the number of high importance features above the drift threshold is greater than 3”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have associated each of Ren’s configured failure criteria with a priority level and to have Ren’s alert indicate that a critical number of criteria above a given priority level is met, as recited in the instant claims, by applying DataRobot’s importance-weighted count rule to Ren’s count-based failure reporting (Ren already triggers on a threshold number of failure instances and already reporl:s “for ML models with relatively high priority values (e.g., satisfying a threshold priority level)” (see Ren para 0109)). As DataRobot itself states: “Particularly, drift in features with high importance can be a warning flag about your model accuracy.” (see DataRobot p. 4, § “Feature Drift vs Feature Importance chart”), which addresses the well-recognized problem of alerting on low-significance deviations, and DataRobot further instructs users to “Exclude features expected to drift from drift status calculation and alerting, so you do not get false alarms.” (see DataRobot p. 21, § “Customize data drift status”) One of ordinary skill in the art of deployed-model monitoring would have recognized that Ren’s count-of-failures trigger treats all failure conditions alike and that DataRobot’s priority-restricted count was a known, predictable refinement to exactly that trigger, requiring no change to Ren’s monitoring, rule delivery, or report transmission. The combination is further supported by MPEP 2143(1) Rationale (C) (use of a known technique to improve similar methods in the same way) and Rationale (G) (DataRobot’s express motivation).The motivation to combine applies to all references under this heading and also therefore arises from DataRobot itself, and one of ordinary skill would have had a reasonable expectation of success because the modification only adds a priority tag and a filtered count to Ren’s existing count-versus-threshold comparison.
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song and further in view of Hashimoto (US 2021/0042592 A1)
Per claim 25, the prior art of record does not explicitly state the algorithm and/or AI model is a first algorithm and/or AI model and wherein the method further comprises: monitoring output data of a compute graph, in particular a second algorithm and/or AI model of the compute graph, the compute graph comprising the first AI model; wherein the one or more performance criteria are determined based on the output data of the compute graph.
Hashimoto et al. (US 2021/0042592 Al) is relied upon to teach the algorithm and/or AI model is a first algorithm and/or AI model (reads on “In the present embodiment, the confidence calculation unit 31 uses, as a classifier, a DNN that has been trained in advance in such a way as to calculate, for each of the plurality of regions on the image, the confidence score for each type of object to be detected”, see Hashimoto para 0037. Hashimoto’s confidence calculation unit uses a trained DNN classifier, which is the first AI model) and wherein the method further comprises: monitoring output data of a compute graph (reads on “The tracking unit 33 then informs the frequency counting unit 34 of the estimated type of the object in each of the images during the tracking period”, see Hashimoto para 0052. The output data of the second algorithm is monitored (counted) - “monitoring output data of a compute graph”), in particular a second algorithm and/or AI model of the compute graph (reads on “The tracking unit 33 tracks, for each object region detected from the newest image, the object represented in the object region by associating the object with an object detected from a past image”, see Hashimoto para 0050. Second algorithm of the graph, a dependent node consuming the DNN-based detections.), the compute graph comprising the first AI model (reads on “The processor 23 includes a confidence calculation unit 31, an object detection unit 32, a tracking unit 33, a frequency counting unit 34, a threshold updating unit 35, a driving planning unit 36, and a vehicle control unit 37”, see Hashimoto para 0035. Connected processing nodes on the same edge processor (vehicle ECU) - a compute graph comprising the DNN.); wherein the one or more performance criteria (reads on “When a confidence score calculated for the type is equal to or higher than the updated confidence threshold for a region included in a predetermined range including the estimated position, the object detection unit 32 then detects the region as the object region in which the object of the type is represented”, see Hashimoto para 0047. Hashimoto then compares the DNN’s confidence scores against the updated, graph-derived threshold, confirming that the graph output determines the criterion actually applied) are determined based on the output data of the compute graph (reads on “The threshold updating unit 35 updates, for each type of the object estimated by the object detection unit 32 during the tracking period for the tracked object, the confidence threshold applied to the type of the object based on the frequency of occurrence for each type of the object”, see Hashimoto para 0054. The confidence threshold applied to the DNN’s output is determined based on the output data of the compute graph (tracker-derived frequencies)).
Before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Hashimoto’s edge-side threshold control, in which a confidence threshold applied to a DNN’s output is updated from the output data of downstream nodes (tracking unit and frequency counting unit) of the same on-device processing graph (see Hashimoto para 0054: ‘The threshold updating unit 35 updates, for each type of the object estimated by the object detection unit 32 during the tracking period for the tracked object, the confidence threshold applied to the type of the object based on the frequency of occurrence for each type of the object”), to realize the instant limitations. One or more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {C, G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have monitored the output data of the compute graph containing Ren’s ML model and to have determined Ren’s confidence-probability failure threshold based on that output data, as recited in the instant claims, by applying Hashimoto’s graph-output-based confidence threshold update to Ren’s confidence-threshold failure rule. As Hashimoto itself states: “Thus, the object detection device can improve estimation accuracy of a type of an object to be detected while suppressing an increase in an amount of calculation and a training man-hour by the classifier.” (see Hashimoto para 0069), which addresses the well-recognized problem of improving how a classifier’s output is evaluated on a resource-constrained edge processor without retraining the classifier or adding computation. One of ordinary skill in the art of edge ML inference would have recognized that Ren’s UE, also a resource-constrained device applying a confidence threshold to its model’s output, would benefit from the same calibration using downstream temporal-consistency information, and that Hashimoto’s tracker-derived[AltContent: ] threshold calibration was a known, predictable technique for doing so; this is a proposed analogous use of the calibration technique, not an assertion that Ren already suffers Hashimoto’s specific object-confusion defect. In the proposed combination, the model manager’s provided rule specifies the confidence-threshold update as a function of the downstream graph output; the platform applies Hashimoto’s calculation under that provided rule and uses the result in Ren’s confidence-based failure comparison and central reporting, with no change to Ren’s comparison and reporting steps. The combination is further supported by MPEP 2143(1) Rationale (C) (use of a known technique to improve similar devices in the same way) and Rationale (G) (Hashimoto’s express motivation). The motivation to combine therefore arises from Hashimoto itself, and one of ordinary skill would have had a reasonable expectation of success because both references apply a confidence threshold to the output of an ML model executing on an edge processor.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Ren in view of Song and further in view of Srinivasan (US 11,352,013)
Per claim 26, the prior art of record does not explicitly state wherein the algorithm and/or AI model is a first algorithm and/or AI model and wherein the one or more performance criteria are determined based on output data of a second algorithm and/or AI model, in particular of a second algorithm and/or AI model deployed on the same or another platform.
Srinivasan (US 11,352,013) is relied upon to teach the algorithm and/or AI model (reads on “For example, the vehicle device may compare the probability output by the one or more machine learning models to a prediction number to determine whether the probability satisfies the prediction number”, see Srinivasan col. 27, ll. 25-29. First AI model on the edge (vehicle) device with an output threshold (prediction number)) is a first algorithm and/or AI model and wherein the one or more performance criteria are determined (reads on “Based on the[AltContent: ] comparison of the output of the vehicle device and the output of the event analysis system, the vehicle device and/or the event analysis system may determine that the vehicle device is identifying false positive events and may adjust the threshold number”, see Srinivasan col. 29, ll. 44-48. Srinivasan again ties the adjustment of the threshold number to the comparison with the second model’s output, here to correct false positives) based on (reads on “The vehicle device can further tune a prediction number (e.g., a prediction threshold) for identifying particular features based on the comparison of the output of the backend server and the output of the vehicle device”, see Srinivasan col. 8, ll. 50-53. The threshold (performance criterion) is determined based on the output data of the second AI model) output data of a second algorithm and/or AI model (reads on “At block 240, the event analysis system can provide the results of the high-fidelity event models to the vehicle device”, see Srinivasan col. 28, ll. 40--42. The backend returns the second model’s results to the edge device, so the edge device has the second model’s output data on which to base its threshold.), in particular of a second algorithm and/or AI model deployed on the same or another platform (reads on “Further, the backend server may implement a more resource-intensive version of the machine learning model (e.g., based on the resources available at the backend server) to provide a more accurate prediction of whether the sensor data is associated with occurrence of the particular feature and/or occurrence of the event”, see Srinivasan col. 24, ll. 34-40. Second AI model deployed on another platform (backend server)).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to further modify the Ren-Song system by integrating Srinivasan’s edge/backend threshold tuning, in which the threshold applied to an edge-deployed ML model’s output is determined based on the output of a second, higher-fidelity ML model deployed on the backend server (see Srinivasan col. 8, ll. 50-53: “The vehicle device can further tune a prediction number (e.g., a prediction threshold) for identifying particular features based on the comparison of the output of the backend server and the output of the vehicle device,”), to realize the instant limitations. One or[AltContent: ] more of the underpinning rationales, as discussed in KSR, also see MPEP 2141, 2143(1) {C,G}, support this conclusion. Accordingly, it would have been obvious to one of ordinary skill in the art to have determined Ren’s confidence-probability failure threshold from the output of Srinivasan’s backend (second) model, as recited in the instant claims, by applying Srinivasan’s second-model-referenced threshold tuning to Ren’s threshold-based failure rule. In the proposed combination, the network-side model manager applies Srinivasan’s comparison-based threshold adjustment and includes the resulting model-specific threshold in the status-reporting configuration delivered to the UE; the UE then applies that provided threshold to its monitored confidence data and sends its model-failure report through Ren’s existing path, so the criterion remains based on the rules provided by the central server. Ren’s on-UE uncertainty model is not relied on as a source of that threshold. The second model is Srinivasan’s backend model, described as “a more resource-intensive version of the machine learning model” (see Srinivasan col. 24, ll. 35-36), whose results are returned to the edge device and used to adjust the threshold. As Srinivasan itself states: “Based on the comparison of the output of the vehicle device and the output of the event analysis system, the vehicle device and/or the event analysis system may determine that the vehicle device is identifying false positive events and may adjust the threshold number.” (see Srinivasan col. 29, ll.44-48), which addresses the well-recognized problem of false positives from a fixed threshold on a resource-constrained edge model. One of ordinary skill in the art of edge-deployed ML would have recognized that Ren’s UE, likewise a resource-constrained device applying a confidence threshold to its model’s output, would benefit from the same calibration, and that Srinivasan’s use of a more accurate second model’s output to set the threshold was a known, predictable solution, requiring no change to Ren’s monitoring or reporting. The combination is further supported by MPEP 2143(1) Rationale (C) and Rationale (G) (Srinivasan’s express motivation). The motivation to combine therefore arises from Srinivasan itself, and one of ordinary skill would have had a reasonable expectation of success because both references compare an edge model’s output probability with a threshold and Srinivasan merely sets that threshold from a second model’s output.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brian Shaw whose telephone number is (571)270-5191. The examiner can normally be reached on Mon-Thurs from 6:00 AM-3:30 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeff Nickerson can be reached on (469) 295-9235. The fax phone number for the organization where this application or proceeding is assigned is 703-872-9306. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN F SHAW/
Primary Examiner, Art Unit 2432
