DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202010480881.7, filed on May 30th, 2020. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Amendment
The amendment filed on March 16th, 2026 has been entered and Claim(s) 1-22 are pending. Claim(s) 5 and 19 have been withdrawn from consideration.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-2, 8, 15-16 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8-13 and 15-18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over McMahan et al., (US20190227980A1), further in view of Tullberg et al., (US20220078637A1).
Regarding claim 1 and analogous claim 15:
McMahan teaches:
An information processing method, applied to a first artificial intelligence (AI) entity in an access network, ([0146], “The network 242 can be any type of communications network, such as a local area network (e.g. intranet), wide area network (e.g. Internet), cellular network, or some combination thereof [access network]. The network 242 can also include a direct connection between a client device 230 and the server 210. In general, communication between the server 210 and a client device 230 can be carried via network interface using any type of wired and/or wireless connection, using a variety of communication protocols”…[0148], “the method (300) can include selecting, by one or more server computing devices, a subset of client computing devices from a pool of available client computing devices”…[0149], “the method (300) can include providing, by the one or more server computing devices, the machine-learned model [first artificial intelligence entity] to the selected client computing devices, and at (306), the method (300) [information processing] can include receiving, by the selected client computing devices, the machine-learned model. In some implementations, the machine-learned model can be, for example, a global set of parameters.”)
the method comprising: receiving, by the first Al entity, second Al model information sent by a terminal device, ([0162], “At (406) the method (400) can include receiving, from each selected client computing device [terminal device], a local update [second AI model information] for the machine-learned model [AI entity]. In some implementations, each local update can be determined based at least in part on a local dataset stored locally on the selected client computing device.”)
wherein the second Al model information does not comprise user data of the terminal device; ([0129], “the training data 108 can be any data derived through a user interaction with a client computing device 102 [terminal device] (i.e., wherein user interaction is interpreted as ‘user data’)”…“For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined”…[0131], “Client computing devices 102 can be configured to provide the local updates to the one or more server computing devices 104. As indicated above, training data 108 may be privacy sensitive. In this manner, the local updates can be performed and provided to server 104 without compromising the privacy of training data 108. For instance, in some implementations, training data 108 is not provided to the one or more server computing devices 104 (i.e., wherein the user data is not provided, hence, ‘does not comprise user data’). Further, in some implementations, the local update [second AI model information] does not include training data 108. In some implementations, one or more of encryption techniques and/or other security techniques can be added to the training process to assist in obscuring any inferable information. In some implementations, the local update can be clipped by the client computing device 102 before being provided to the one or more server computing devices 104, as disclosed herein.”)
updating, by the first Al entity, first Al model information based on the second Al model information, ([0132], “the one or more server computing devices 104 (i.e., wherein the first AI entity is interpreted as the machine learning model) can receive each local update [second AI model information] from client device 102, and can determine differentially private aggregate of the local updates. For example, in some implementations, a bounded-sensitivity data-weighted average of the local updates can be determined. Further, the one or more server computing devices 104 can determine an updated machine-learned model [first AI model information] based at least in part on the differentially private aggregate of the local updates.”)
wherein the first Al model information is Al model information of the first Al entity; and ([0150], “determining, by each selected client computing device, a local update based at least in part on a local dataset stored locally on the selected client computing device. For example, in some implementations, each selected client computing device can train the machine-learned model [first AI entity] based at least in part on the local dataset to determine a locally-trained model [first AI model information]”)
sending, by the first Al entity, the updated first Al model information to the terminal device ([0157], “providing, by the one or more server computing devices [first AI entity], the updated machine-learned model [updated first AI model information] to one or more client computing devices, and at (320), the method (300) can include receiving, by the one or more client computing devices [terminal device], the updated machine-learned model.”)
McMahan does not explicitly teach:
receiving, by the first Al entity, Al information of the terminal device that comprises an Al update parameter, the Al update parameter having a first value that indicates a scheduled Al update or a second value that indicates an event-triggered Al update; and
receiving, by the first Al entity, feedback information based on the Al update parameter indicating the scheduled Al update or the event-triggered Al update, wherein the feedback information indicates data used for Al training
Tullberg teaches:
receiving, by the first Al entity, Al information of the terminal device that comprises an Al update parameter, ([0014]-[0015], “The network node receives information from the wireless device [terminal device] (i.e., wherein terminal device under the broadest reasonable interpretation is interpreted as wireless device). The information relates to at least one prediction of an operation of the wireless device and to at least one result of the operation. The at least one prediction of the operation is obtained by means of the first instance of the machine learning model (i.e., wherein first AI entity under the broadest reasonable interpretation is interpreted as first instance), and the operation is relating to a transmission over the communications interface. The network node then updates one or more parameters [AI update parameter] of the second instance of the machine learning model based on the received information.”)
the Al update parameter having a first value that indicates a scheduled Al update or a second value that indicates an event-triggered Al update; and ([0085], “The network node 110, 130, 143 may transmit the request when a period of time has expired [scheduled update] (i.e., wherein scheduled ai update under the broadest reasonable interpretation is interpreted as expiration of time is a scheduled occurrence), when a number of received user communications from the wireless device 120 is above a threshold value for the user communications [event triggered] (i.e., wherein above a threshold value under the broadest reasonable interpretation is interpreted as a trigger); and/or when an error in the at least one prediction of the operation is expected.”)
receiving, by the first Al entity, feedback information based on the Al update parameter indicating the scheduled Al update or the event-triggered Al update, ([0206]-[0207], “the wireless device 120 transmits the ML training errors to the network node 110, 130, 143, cf. Action 803 (i.e., wherein errors under the broadest reasonable interpretation is interpreted as feedback information). The AP receives the errors, updates in Action 804 the ML model, and determines if an updated version of W should be transmitted to e.g. the wireless device. If the updates to the ML model parameters are too small to merit transmission (i.e., wherein based on ai update parameter), a “no update indication” is sent to the wireless device 120, cf. Action 804. However, the updated instance of the model is kept in the network node 110, 130, 143 and subsequent ML model updates are performed by the network node 110, 130, 143 on the most current instance of the ML model. When eventually the updates are sufficiently different compared to the first instance of the ML model in the wireless device 120, the updated W, sometimes in this disclosure referred to as W*, is transmitted to the wireless device 120, cf. Action 804. The network node 110, 130, 143 may also trigger the transmission of ML training errors,”…[0085], “The network node 110, 130, 143 may transmit the request when a period of time has expired [scheduled update] (i.e., wherein scheduled ai update under the broadest reasonable interpretation is interpreted as expiration of time is a scheduled occurrence), when a number of received user communications from the wireless device 120 is above a threshold value for the user communications [event triggered] (i.e., wherein above a threshold value under the broadest reasonable interpretation is interpreted as a trigger); and/or when an error in the at least one prediction of the operation is expected.”)
wherein the feedback information indicates data used for Al training ([0015]-[0016], “The network node then updates one or more parameters of the second instance of the machine learning model based on the received information. Further, the network node transmits information relating to the updated one or more parameters of the second instance of the machine learning model to the wireless device,”… ([0206]-[0207], “the wireless device 120 transmits the ML training errors to the network node 110, 130, 143, cf. Action 803 (i.e., wherein errors under the broadest reasonable interpretation is interpreted as feedback information). The AP receives the errors, updates in Action 804 the ML model, and determines if an updated version of W should be transmitted to e.g. the wireless device. If the updates to the ML model parameters are too small to merit transmission (i.e., wherein based on ai update parameter, hence used for ai training))
Tullberg and McMahan are both related to the same field of endeavor (i.e., machine learning). In view of the teachings of Tullberg it would have been obvious for a person of ordinary skill in the art to apply the teachings of Tullberg to McMahan before the effective filing date of the claimed invention in order to improve the efficiency of resource management (Tullberg, [0052], “it is provided a way of improving the performance in the wireless communications system by e.g. improving usage of resources in the wireless communications system. However, even if some embodiments described herein relate to improved resource utilization it should be understood that some embodiments disclosed herein, alternatively or additionally, may provide an improved flexibility and/or an improved adaptability.”)
Regarding claim 2 and analogous claim 16:
McMahan, as modified by Tullberg teaches the method of claim 1.
McMahan further teaches:
wherein the method further comprises: receiving, by the first Al entity, a request message sent by the terminal device, ([0133], “The system 200 can be implemented using a client-server architecture that includes a server 210 (e.g., one or more server computing devices) [first AI entity] that communicates with one or more client devices 230 (e.g., client computing devices) [terminal device] over a network 242. Thus, FIG. 2 provides an example system 200 that can implement the scheme illustrated by system 100 of FIG. 1. (i.e., wherein client-server architecture under the broadest reasonable interpretation (BRI) is interpreted as ‘a request message sent’)”)
wherein the request message requests the first Al model information; and sending, by the first Al entity, the first Al model information to the terminal device ([0133], “The system 200 can be implemented using a client-server architecture that includes a server 210 [first AI entity] (e.g., one or more server computing devices) that communicates with one or more client devices 230 [terminal device] (e.g., client computing devices) over a network 242” (i.e., wherein the request message)…[0149], “providing, by the one or more server computing devices, the machine-learned model [first AI model information] to the selected client computing devices, and at (306), the method (300) can include receiving, by the selected client computing devices, the machine-learned model. In some implementations, the machine-learned model can be, for example, a global set of parameters.”)
Regarding claim 3 and analogous claim 17:
McMahan teaches the method of claim 2.
McMahan does not explicitly teach:
wherein before receiving, by the first Al entity, the request message sent by the terminal device, the method further comprises: receiving, by the first Al entity, second Al information of the terminal device, wherein the second Al information of the terminal device comprises an Al capability parameter
Tullberg teaches:
wherein before receiving, by the first Al entity, the request message sent by the terminal device, ([0188], “a request for the information relating to the at least one prediction of the operation of the wireless device 120 [terminal device] and to the at least one result of the operation. In such embodiments, the wireless device 120 may be configured to transmit, to the network node 110, 130, 143 [first AI entity], the information in response to the received request (i.e., wherein before receiving is interpreted as in response to the received request)”
the method further comprises: receiving, by the first Al entity, second Al information of the terminal device, wherein the second Al information of the terminal device comprises an Al capability parameter ([0191], “The information may relate to the machine learning model [AI information], such as the second instance of the machine learning model, information received from the network node 110, 130, 143, and information transmitted to the network node 110”…“The memory is arranged to be used to store obtained information, data, configurations, and applications etc. to perform the methods herein when being executed in the wireless device 120. the communications system 10 comprises a network node 110, 130, 143, e.g. an Access Point (AP) such as an eNB [first AI entity], and two wireless devices 120, 122 of different machine learning capabilities. The eNB is connected to a core network, e.g. the core network 102, and possibly a cloud infrastructure, such as a computer cloud 140”…[0196], “The wireless devices attached to the eNB may be of different machine learning capabilities, such as a first wireless device with capability for ML training, and a second wireless device with limited capability for ML training. A first wireless device, e.g. the wireless device 120, may be a smart phone with capability of ML training and a second wireless device, e.g. the wireless device 122, may be a connected temperature sensor with limited capabilities for ML training (i.e., wherein the AI information from the wireless devices ‘terminal device’ comprises capability for machine learning)”)
Tullberg and McMahan are both related to the same field of endeavor (i.e., machine learning). In view of the teachings of Tullberg it would have been obvious for a person of ordinary skill in the art to apply the teachings of Tullberg to McMahan before the effective filing date of the claimed invention in order to improve the efficiency of resource management (Tullberg, [0052], “it is provided a way of improving the performance in the wireless communications system by e.g. improving usage of resources in the wireless communications system. However, even if some embodiments described herein relate to improved resource utilization it should be understood that some embodiments disclosed herein, alternatively or additionally, may provide an improved flexibility and/or an improved adaptability.”)
Regarding claim 4 and analogous claim 18:
McMahan, as modified by Tullberg, teaches the method of claim 3.
McMahan does not explicitly teach:
wherein in response to the Al capability parameter indicating that the terminal device has an Al inference capability, the first Al entity receives Al decision information and status information sent by the terminal device, wherein the Al decision information is obtained by the terminal device by inputting the status information into a second Al model for inference, the status information is obtained by the terminal device based on observation information, and the observation information indicates data used for Al decision.
Tullberg further teaches:
wherein in response to the Al capability parameter indicating that the terminal device has an Al inference capability, ([0009], “disclosed herein enables training of a machine learning model at a network node that is located remotely from a wireless device [terminal device] that is using the machine learning model to perform predictions. The wireless device may have limited machine learning capabilities and thus it may be unable to perform training of the machine learning model itself but the machine learning model may be trained by the network node having more machine learning capabilities. In this case, the wireless device needs to transmit relevant training data to the network node. By the expression “network node with more ML capabilities” when used in this disclosure is meant a network node that have sufficient processing and storing capabilities to perform machine learning, e.g. more ML capabilities than the wireless device (i.e., wherein once the terminal device communicates it has AI capabilities (i.e., inference)). For example, the network node with more ML capabilities is a network node having capability of doing the ML inference [AI inference capability]”)
the first Al entity receives Al decision information and status information sent by the terminal device, ([0088], “The network node 110, 130, 143 [first AI entity] receives, from the wireless device 120 [terminal device], information relating to at least one prediction [AI decision information] of an operation of the wireless device 120 and to at least one result [status information] of the operation. The at least one prediction of the operation is obtained by means of the first instance of the machine learning model. The at least one prediction of the operation may be obtained or determined by the wireless device 120”)
wherein the Al decision information is obtained by the terminal device by inputting the status information into a second Al model for inference, ([0076], “the wireless device 120 [terminal device] performs the inference, i.e. the prediction [AI decision information], using a first instance [status information] of the machine learning model (i.e., wherein terminal device uses status information and performs a second AI model for inference is interpreted as the first model is performed at the first entity). For example, the wireless device 120 performs the prediction by performing a forward propagation in case of a neural network.”)
the status information is obtained by the terminal device based on observation information, and the observation information indicates data used for Al decision ([0089], “The at least one prediction of the operation is obtained by means of the first instance of the machine learning model. The at least one prediction [AI decision] of the operation may be obtained or determined by the wireless device 120 [terminal device] by means of the first instance [status information] of the machine learning model. Further, the at least one result of the operation may be obtained or determined by the wireless device 120 by performing the operation. Further, the operation is relating to a transmission [observation information] over the communications interface. For example, the operation may be a beam operation such as an operation to change transmit beam and/or receive beam for a transmission to be transmitted or received by the wireless device 120”)
The motivation for claim 4 is the same as for claim 3.
Regarding claim 6 and analogous claim 20:
McMahan, as modified by Tullberg, teaches the method of claim 1.
McMahan does not explicitly teach:
wherein the method further comprises: updating, by the first Al entity, a first Al model based on Al training data, wherein the Al training data comprises one or more of Al decision information, status information, or the feedback information
Tullberg further teaches:
wherein the method further comprises: updating, by the first Al entity, a first Al model based on Al training data, wherein the Al training data comprises one or more of Al decision information, status information, or the feedback information ([0207], “the network node [first AI entity] 110, 130, 143 may trigger a transmission from multiple wireless devices and update a common model based on the collective error messages [AI training data] (i.e., wherein AI training data is interpreted under the broadest reasonable interpretation (BRI) to include status and/or feedback information) from all wireless devices such as from all sensors, cf. Action 806. The updated model W may then be transmitted to the wireless devices where the updates are sufficiently large, cf. Action 807.”)
The motivation for claim 6 is the same as for claim 3.
Regarding claim 8:
McMahan teaches:
An information processing method, comprising: sending, by a terminal device, a request message to a first artificial intelligence (AI) entity, ([0148], “the method (300) can include selecting, by one or more server computing devices, a subset of client computing devices from a pool of available client computing devices [terminal device]”…[0149], “the method (300) can include providing, by the one or more server computing devices, the machine-learned model [artificial intelligence entity] to the selected client computing devices, and at (306), the method (300) [information processing] can include receiving, by the selected client computing devices, the machine-learned model. In some implementations, the machine-learned model can be, for example, a global set of parameters.”)
wherein the request message requests first Al model information; receiving, by the terminal device, the first Al model information sent by the first Al entity; ([0133], “The system 200 can be implemented using a client-server architecture that includes a server 210 [first AI entity] (e.g., one or more server computing devices) that communicates with one or more client devices 230 [terminal device] (e.g., client computing devices) over a network 242” (i.e., wherein the request message)…[0149], “providing, by the one or more server computing devices, the machine-learned model [first AI model information] to the selected client computing devices, and at (306), the method (300) can include receiving, by the selected client computing devices, the machine-learned model. In some implementations, the machine-learned model can be, for example, a global set of parameters.”)
McMahan does not explicitly teach:
inputting, by the terminal device, status information into a second Al model for inference, to obtain Al decision information of the terminal device, wherein the status information is determined based on observation information, indicating data used for Al decision, wherein the second Al model is determined by the terminal device based on the first Al model information, and
Tullberg teaches:
inputting, by the terminal device, status information into a second Al model for inference, to obtain Al decision information of the terminal device, ([0076], “the wireless device 120 [terminal device] performs the inference, i.e. the prediction [AI decision information], using a first instance [status information] of the machine learning model (i.e., wherein terminal device uses status information and performs a second AI model for inference is interpreted as ‘obtain AI decision information’).”)
wherein the status information is determined based on observation information, indicating data used for Al decision, ([0089], “The at least one prediction of the operation is obtained by means of the first instance of the machine learning model. The at least one prediction [AI decision] of the operation may be obtained or determined by the wireless device 120 [terminal device] by means of the first instance [status information] of the machine learning model. Further, the at least one result of the operation may be obtained or determined by the wireless device 120 by performing the operation. Further, the operation is relating to a transmission [observation information] over the communications interface. For example, the operation may be a beam operation such as an operation to change transmit beam and/or receive beam for a transmission to be transmitted or received by the wireless device 120”)
wherein the second Al model is determined by the terminal device based on the first Al model information and ([0076], “the wireless device [terminal device] 120 performs the inference [second AI model], i.e. the prediction, using a first instance of the machine learning model [first AI model information]. For example, the wireless device 120 performs the prediction by performing a forward propagation in case of a neural network…[0077], “The network node 110, which may be a cloud entity, keeps a second instance of the machine learning model used in the wireless device 120.”)
sending, by the terminal device, feedback information to the first AI entity in response to an AI update parameter indicating a scheduled AI update or an event-triggered AI update, wherein the feedback information indicates data used for AI training, the AI update parameter having a first value that indicates a scheduled AI update or a second value that indicates an event-triggered AI update. ([0206]-[0207], “the wireless device 120 transmits the ML training errors to the network node 110, 130, 143, cf. Action 803 (i.e., wherein errors under the broadest reasonable interpretation is interpreted as feedback information). The AP receives the errors, updates in Action 804 the ML model, and determines if an updated version of W should be transmitted to e.g. the wireless device. If the updates to the ML model parameters are too small to merit transmission, a “no update indication” is sent to the wireless device 120, cf. Action 804. However, the updated instance of the model is kept in the network node 110, 130, 143 and subsequent ML model updates are performed by the network node 110, 130, 143 on the most current instance of the ML model. When eventually the updates are sufficiently different compared to the first instance of the ML model in the wireless device 120, the updated W, sometimes in this disclosure referred to as W*, is transmitted to the wireless device 120, cf. Action 804. The network node 110, 130, 143 may also trigger the transmission of ML training errors,”…[0085], “The network node 110, 130, 143 may transmit the request when a period of time has expired [scheduled update] (i.e., wherein scheduled ai update under the broadest reasonable interpretation is interpreted as expiration of time is a scheduled occurrence and first value is interpreted to the time expiration condition), when a number of received user communications from the wireless device 120 is above a threshold value for the user communications [event triggered] (i.e., wherein above a threshold value under the broadest reasonable interpretation is interpreted as a trigger and second value is interpreted to the event triggered); and/or when an error in the at least one prediction of the operation is expected.”)
The motivation for claim 8 is the same as for claim 1.
Regarding claim 9:
McMahan, as modified by Tullberg, teaches the method of claim 8.
McMahan does not explicitly teach:
wherein before sending, by the terminal device, the request message to the first Al entity, the method further comprises: sending, by the terminal device, Al information of the terminal device to the first Al entity, wherein the Al information of the terminal device comprises an Al capability parameter, indicating that the terminal device has an Al inference capability
Tullberg teaches:
wherein before sending, by the terminal device, the request message to the first Al entity, the method further comprises: ([0188], “a request for the information relating to the at least one prediction of the operation of the wireless device 120 [terminal device] and to the at least one result of the operation. In such embodiments, the wireless device 120 may be configured to transmit, to the network node 110, 130, 143 [first AI entity], the information in response to the received request (i.e., wherein under the broadest reasonable interpretation, sending a request to check for capability is done before sending the actual request)”
sending, by the terminal device, Al information of the terminal device to the first Al entity, wherein the Al information of the terminal device comprises an Al capability parameter, indicating that the terminal device has an Al inference capability ([0009], “disclosed herein enables training of a machine learning model at a network node that is located remotely from a wireless device [terminal device] that is using the machine learning model to perform predictions. The wireless device may have limited machine learning capabilities and thus it may be unable to perform training of the machine learning model itself but the machine learning model may be trained by the network node having more machine learning capabilities. In this case, the wireless device needs to transmit relevant training data [AI information] to the network node. By the expression “network node with more ML capabilities” when used in this disclosure is meant a network node that have sufficient processing and storing capabilities to perform machine learning, e.g. more ML capabilities than the wireless device (i.e., wherein once the terminal device communicates it has AI capabilities (i.e., inference)). For example, the network node with more ML capabilities is a network node having capability of doing the ML inference [AI inference capability] (i.e., wherein AI capability parameter is interpreted as having AI inference capability)”)
The motivation for claim 9 is the same as for claim 1.
Regarding claim 10:
McMahan, as modified by Tullberg, teaches the method of claim 8.
McMahan does not explicitly teach:
wherein the method further comprises: sending, by the terminal device, the Al decision information and the status information to the first Al entity
Tullberg further teaches:
wherein the method further comprises: sending, by the terminal device, the Al decision information and the status information to the first Al entity ([0095], “based on the received information relating to the at least one prediction of the operation [AI decision information] of the wireless device 120 and to the at least one result of the operation [status information], the network node [first AI entity] 110, 130, 143 determines a training difference between the at least one prediction of the operation of the wireless device 120 [terminal device] and the at least one result of the operation.”)
The motivation for claim 10 is the same as for claim 1.
Regarding claim 11:
McMahan, as modified by Tullberg, teaches the method of claim 8.
McMahan does not explicitly teach:
wherein the feedback information is comprised in AI information of the terminal device, wherein the AI information of the terminal device further comprises an AI capability parameter
Tullberg further teaches:
wherein the feedback information is comprised in AI information of the terminal device, wherein the AI information of the terminal device further comprises an Al capability parameter; (([0206]-[0207], “the wireless device 120 transmits the ML training errors to the network node 110, 130, 143, cf. Action 803 (i.e., wherein errors under the broadest reasonable interpretation is interpreted as feedback information). The AP receives the errors, updates in Action 804 the ML model, and determines if an updated version of W should be transmitted to e.g. the wireless device. If the updates to the ML model parameters are too small to merit transmission, a “no update indication” is sent to the wireless device 120, cf. Action 804”…[0196], “The wireless devices attached to the eNB may be of different machine learning capabilities, such as a first wireless device with capability for ML training, and a second wireless device with limited capability for ML training. A first wireless device, e.g. the wireless device 120, may be a smart phone with capability of ML training and a second wireless device, e.g. the wireless device 122, may be a connected temperature sensor with limited capabilities for ML training [AI capability parameter])
The motivation for claim 11 is the same as for claim 1.
Regarding claim 12:
McMahan, as modified by Tullberg, teaches the method of claim 8.
McMahan does not explicitly teach:
wherein the method further comprises: obtaining, by the terminal device, second Al model information based on Al training data in response to an Al capability parameter indicating that the terminal device has an Al training capability, wherein the Al training data comprises one or more of the Al decision information, the status information, or the feedback information
Tullberg further teaches:
wherein the method further comprises: obtaining, by the terminal device, second Al model information based on Al training data in response to an Al capability parameter indicating that the terminal device has an Al training capability, ([0076], “the wireless device 120 [terminal device] performs the inference, i.e. the prediction, (i.e., wherein under the broadest reasonable interpretation (BRI) the device performing the inference is interpreted as an indication that the device has AI training capability) using a first instance of the machine learning model. For example, the wireless device 120 performs the prediction by performing a forward propagation in case of a neural network (i.e., wherein the performs the inference on the terminal device is interpreted as the second AI model that is using the instance of the AI training data)”)
wherein the Al training data comprises one or more of the Al decision information, the status information, or the feedback information ([0207], “the network node [first AI entity] 110, 130, 143 may trigger a transmission from multiple wireless devices and update a common model based on the collective error messages [AI training data] (i.e., wherein AI training data is interpreted under the broadest reasonable interpretation (BRI) to include status and or feedback information) from all wireless devices such as from all sensors, cf. Action 806. The updated model W may then be transmitted to the wireless devices where the updates are sufficiently large, cf. Action 807.”)
The motivation for claim 12 is the same as for claim 3.
Regarding claim 13:
McMahan, as modified by Tullberg, teaches the method of claim 12.
McMahan does not explicitly teach:
wherein the method further comprises: sending, by the terminal device, the second Al model information to the first Al entity; and
receiving, by the terminal device, updated first Al model information sent by the first Al entity, wherein the updated first Al model information is determined by the first Al entity based on the second Al model information
Tullberg further teaches:
wherein the method further comprises: sending, by the terminal device, the second Al model information to the first Al entity; and ([0022], “wireless device transmits, to the network node [first AI entity], information relating to at least one prediction of an operation of the wireless device and to at least one result of the operation. The at least one prediction of the operation is obtained by means of the first instance of the machine learning model [second AI model]”)
receiving, by the terminal device, updated first Al model information sent by the first Al entity, wherein the updated first Al model information is determined by the first Al entity based on the second Al model information ([0022], “wireless device transmits, to the network node, information relating to at least one prediction of an operation of the wireless device and to at least one result of the operation. The at least one prediction of the operation is obtained by means of the first instance of the machine learning model [second AI model]”…[0076], “The wireless device [terminal device] 120 transmits the training differences are transmitted to the network node [first AI entity]”…[0077], ”the network node 110, 130, 143 trains the second instance of the machine learning model based on the received one or more training differences. Information relating [AI model information] (i.e., wherein the information is interpreted as the updated AI model information) to the updated second instance of the machine learning model is then transmitted back to the wireless device 120, whereby the wireless device 120”)
The motivation for claim 13 is the same as for claim 1.
Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over McMahan et al., as modified by Tullberg et al., further in view of Lei et al., Non-Patent Literature (“Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges”)
Regarding claim 7 and analogous claim 14:
McMahan, as modified by Tullberg, teaches the method of claim 6.
McMahan does not explicitly teach:
wherein the feedback information comprises reward information, and wherein updating the first AI model based on the AI training data comprises updating the first AI model based on the reward information, wherein the reward information is determined based on a reward function, the reward function is determined based on a target parameter θ and a weight value Φ of the target parameter, the target parameter is performance data obtained by the terminal device by executing Al decision information, and the weight value of the target parameter is determined by the first Al entity based on performance data of one or more terminal devices.
Tullberg further teaches:
the reward function is determined based on a target parameter θ and a weight value Φ of the target parameter, ([0075], “During training of a machine learning model, the output of the machine learning model is compared to a known target value. The target is a known value given from the outside and used for training [target parameter]. For example, the target value may be received from a manually designed receiver circuit or from an algorithm. The difference between the output of the machine learning model and the target value is used for training of the machine learning model. This difference is sometimes in this disclosure referred to as training difference [reward function]”…[0102], “when the machine learning model is based on a neural network, the one or more parameters may be one or more weights [weight value] in the neural network. In a neural network implementation, the one or more parameters may be one or more layers, one or more neurons per layer, and/or one or more activations functions of the neural network.”)
the target parameter is performance data obtained by the terminal device by executing Al decision information, and ([0076], “In some embodiments, the wireless device 120 performs the inference [executing AI decision information], i.e. the prediction, using a first instance of the machine learning model. For example, the wireless device 120 [terminal device] performs the prediction by performing a forward propagation in case of a neural network. The wireless device 120 transmits the training differences [target parameter is performance data] are transmitted to the network node 110, 130, 143 or another network entity capable of training of a machine learning model.”)
the weight value of the target parameter is determined by the first Al entity based on performance data of one or more terminal devices ([0102], “when the machine learning model [first AI entity] is based on a neural network, the one or more parameters may be one or more weights [weight value] in the neural network. In a neural network implementation, the one or more parameters may be one or more layers, one or more neurons per layer, and/or one or more activations functions of the neural network.”)
Lei teaches:
wherein the feedback information comprises reward information, and wherein updating the first AI model based on the AI training data comprises updating the first AI model based on the reward information, (Page 7, Col 1, paragraph 1, “In a state st, the agent selects an action at according to the current policy πθ given by the actor network [first AI model], receives a reward rt+1 [reward information] (i.e., wherein the feedback information is interpreted as the reward information)”…(Page 7, Col 1, paragraph 3, “Therefore, given the value functions evaluated by the critic network, the value of G(τst ) − b(st) in (15) can be replaced by δt in (17), which can be seen as an estimate of the advantage of action at in state st [37]. The loss function of the actor network can be defined similar to (15), i.e., L Actor(θ) = −δt log π (at|st; θ). (19) Similar to (10) in the policy gradient method, the parameters of the actor network are updated as θ ← θ + αδt∇θ log π(at|st; θ), (20) where α is the learning rate for the actor.”)
wherein the reward information is determined based on a reward function (Page 5, Col 2, paragraph 2, “To evaluate the performance of the current policy, the objective function [reward function] is defined as J(θ) = V πθ (s0) = Eτs0∼πθ [G(τs0 )] , ∀s0 ∈ S, (7) where V πθ (s0) is the value function of policy πθ as shown in (24), and τs0 refers to the sampling trajectory with an initial state s0. If we can find the parameters θ for policy πθ so that the objective function J(θ) is maximized, we can solve the problem. The basic idea of policy gradient methods are to adjust the parameters in the direction of greater expected reward [reward information].”)
Lei and McMahan are both related to the same field of endeavor (i.e., machine learning). In view of the teachings of Lei it would have been obvious for a person of ordinary skill in the art to apply the teachings of Lei to McMahan before the effective filing date of the claimed invention in order to improve the efficiency of resource allocation by using reinforcement learning (Lei, Abstract, “In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making.”)
Claim(s) 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over McMahan et al., as modified by Tullberg et al., further in view of Mirhoseini et al., Non-Patent Literature (“Chip Placement with Deep Reinforcement Learning”)
Regarding claim 21:
McMahan, as modified by Tullberg, teaches the method of claim 1.
McMahan does not explicitly teach:
wherein the feedback information comprises reward information, the method further comprising: receiving, by the first AI entity, reward function update indication information; and updating, by the first AI entity, a reward function based on the reward information, wherein the reward function computes a reward for updating first AI model information, and wherein the reward function is based on performance data of the terminal device or performance data of the terminal device and one or more further terminal devices.
Tullberg further teaches:
wherein the feedback information comprises reward information, the method further comprising: receiving, by the first AI entity, reward function update indication information; and ([0206]-[0207], “the wireless device 120 transmits the ML training errors to the network node 110, 130, 143, cf. Action 803 (i.e., wherein errors under the broadest reasonable interpretation is interpreted as feedback information). The AP receives the errors, updates in Action 804 the ML model, and determines if an updated version of W should be transmitted to e.g. the wireless device. If the updates to the ML model parameters are too small to merit transmission, a “no update indication” (i.e., wherein update indication information) is sent to the wireless device 120, cf. Action 804”…[0196], “The wireless devices attached to the eNB may be of different machine learning capabilities, such as a first wireless device with capability for ML training, and a second wireless device with limited capability for ML training. A first wireless device, e.g. the wireless device 120, may be a smart phone with capability of ML training and a second wireless device, e.g. the wireless device 122, may be a connected temperature sensor with limited capabilities for ML training”)
Mirhoseini teaches:
updating, by the first AI entity, a reward function based on the reward information, wherein the reward function computes a reward for updating first AI model information, and (Page 4, col 1, para 3, “The action space is all valid placements of the tth macro, which is a function of the density mask described in section 3.3.6. Action at is the cell placement of the tth macro that was chosen by the RL policy network. st+1 is the next state, which includes an updated representation containing information about the newly placed macro, an updated density mask, and an embedding for the next node to be placed. In our formulation, rt is0foreverytimestepexcept for the final rT, where it is a weighted sum of approximate wire length and congestion (i.e., reward function computes a reward) as described in Section3.3. Through repeated episodes (sequences of states, actions, and rewards) (i.e., reward information) the policy network learns to take actions that will maximize cumulative reward. We use Proximal Policy Optimization(PPO) to update the parameters of the policy network (i.e., wherein updating the first AI model), given the cumulative reward for each placement.”)
wherein the reward function is based on performance data of the terminal device or performance data of the terminal device and one or more further terminal devices (Section 3.3, “in this work is to minimize power, performance and area, subject to constraints on routing congestion and density. Our true reward is the output of a commercial EDA tool, including wire length (i.e., wherein reward function ‘true reward’), routing congestion, density, power, timing, and area (i.e., wherein performance data of the terminal device). However, RL policies require 100,000 soft examples to learn effectively, so it is critical that the reward function be fast to evaluate, ideally running in a few milli seconds. In order to be effective, these approximate reward functions must also be positively correlated with the true reward. Therefore, a component of four cost is wire length, because it is not only much cheaper to evaluate, but also correlates with power and performance (timing).”)
Mirhoseini and McMahan are both related to the same field of endeavor (i.e., machine learning). In view of the teachings of Mirhoseini it would have been obvious for a person of ordinary skill in the art to apply the teachings of Mirhoseini to McMahan before the effective filing date of the claimed invention in order to improve the quality of the training information used to update the AI model (Mirhoseini, Introduction, Col 2, “To address this challenge, we pose chip placement as a Reinforcement Learning (RL) problem, where we train an agent (e.g., RL policy network) to optimize the placements.”)
Regarding claim 22:
McMahan, as modified by Tullberg, teaches the method of claim 6.
McMahan, as modified by Tullberg does not explicitly teach:
further comprising: performing mask processing on the AI decision information, based on decision mask information, to obtain masked AI decision information, wherein the updating, by the first AI entity, the first AI model based on the AI training data comprises updating the first AI model based on the masked AI decision information.
Mirhoseini teaches:
further comprising: performing mask processing on the AI decision information, based on decision mask information, to obtain masked AI decision information, (Section 3.3.6 para 2, “during each RL step, we calculate the current density mask (i.e., wherein decision mask information), a binary m × n matrix that represents grid cells onto which we can place the center of the current node without violating the density threshold criteria. Before choosing an action from the policy network output (i.e., wherein AI decision information), we first take the dot product of the mask (i.e., wherein performing mask processing)”)
wherein the updating, by the first AI entity, the first AI model based on the AI training data comprises updating the first AI model based on the masked AI decision information (Page 4, col 1, para 3, “The action space is all valid placements of the tth macro, which is a function of the density mask described in section 3.3.6. Action at is the cell placement of the tth macro that was chosen by the RL policy network. st+1 is the next state, which includes an updated representation containing information about the newly placed macro, an updated density mask (i.e., wherein masked ai decision information), and an embedding for the next node to be placed. In our formulation, rt is0foreverytimestepexcept for the final rT, where it is a weighted sum of approximate wire length and congestion as described in Section3.3. Through repeated episodes (sequences of states, actions, and rewards) (i.e., wherein ai training data), the policy network learns to take actions that will maximize cumulative reward. We use Proximal Policy Optimization(PPO) to update the parameters of the policy network (i.e., wherein updating the first AI model), given the cumulative reward for each placement.”)
Mirhoseini and McMahan are both related to the same field of endeavor (i.e., machine learning). In view of the teachings of Mirhoseini it would have been obvious for a person of ordinary skill in the art to apply the teachings of Mirhoseini to McMahan before the effective filing date of the claimed invention in order to improve the quality of the training information used to update the AI model (Mirhoseini, Introduction, Col 2, “To address this challenge, we pose chip placement as a Reinforcement Learning (RL) problem, where we train an agent (e.g., RL policy network) to optimize the placements.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMINA BENOURAIDA whose telephone number is (571)272-4340. The examiner can normally be reached Monday-Friday 8:30am-5pm ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMINA MORENO BENOURAIDA/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights