DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is in response to Application 18/893,780 filed on 7 March, 2025.
This application is a continuation of International Application No. PCT/EP2022/057740, filed on 24 March, 2022.
Claims 1-17 are pending.
Claim Rejections - 35 USC § 112
Claim 5-6 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The claims are generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors.
Claim 5 recites “wherein the first model comprises multiple parts and one intermediate output of the one or more intermediate outputs comprises an output of a first part of the multiple parts of the first model”. Examiner respectfully requests clarification with pertinent support from the original disclosure as filed for this limitation so that the limitation can be understood and prior art applied, if applicable.
Dependent claim 6 does not cure the deficiency of parent claim 5 and therefore, inherits the rejection.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4 and 7-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Distributed Deep Neural Networks over the Cloud, the Edge and End Devices” by Teerapittayanon et al, dated 2017, hereinafter NPL.
Regarding claim 1, NPL teaches a device (NPL section III. A: “by performing a portion of the DNN inference computation on the device rather than sending the raw input to the cloud”) for processing a data sample to form a predicted output (NPL section III. A: “The configurations presented show how DDNN can scale the inference computation across different physical devices”; inference meaning that a prediction is generated based on the data input), the device comprising a processor, wherein the device is configured to:
receive the data sample (NPL section I: "that capture a large quantity of input data in a streaming fashion” being implied that for inference data has to be received);
input the data sample and/or one or more of any intermediate outputs derived from the data sample to a learnable control function (NPL section III.D: “We use a normalized entropy threshold as the confidence criteria”, which uses intermediate outputs “C is the set of all possible labels and x is a probability vector”);
and in dependence on an output of the learnable control function (NPL section III. D: “This normalized entropy n has values between 0 and 1 which allows easier interpretation and searching of its corresponding threshold T”) perform one of the following:
(i) process the data sample to form the predicted output using a first model stored locally at the device (NPL section III. A: “Using an exit point after device inference, we may classify those samples which the local network is confident about, without sending any information to the cloud”, wherein the model used is stored in the device); and/or
(ii) send the data sample and/or the one or more of any intermediate outputs derived from the data sampleFor more difficult cases, the intermediate DNN output (up to the local exit) is sent to the cloud, where further inference is performed using additional NN layers and a final classification decision is made”).
Regarding claim 2, the device of claim 1, wherein, processing the data sample to form the predicted output using the first model stored locally at the device is in response to the output of the learnable control function exceeding a threshold (NPL section III. D: “At each exit point, n is computed and compared against T in order to determine if the sample should exit at that point”, meaning that the exit point prediction is used; “At a given exit point, if the predictor is not confident in the result (i.e., n > T), the system falls back to a higher exit point in the hierarchy until the last exit is reached which always performs classification”; Section D).
Regarding claim 3, the device of claim 1, wherein sending the data sample and/or the one or more of the any intermediate outputs derived from the data sample to the remote location for the input to the second model stored at the remote location to form the predicted output is in response to the output of the learnable control function not exceeding a threshold (NPL section D, F).
Regarding claim 4, the device of claim 1, wherein the one or more of the any intermediate outputs derived from the data sample are one or more intermediate outputs of the first model stored at the device (NPL section III. D: “We use a normalized entropy threshold as the confidence criteria”, which consists in the uncertainty of the early exit prediction and that intermediate outputs are sent as an input to a remote location if necessary; section III. A: “For more difficult cases, the intermediate DNN output (up to the local exit) is sent to the cloud, where further inference is performed using additional NN layers and a final classification decision is made”).
Regarding claim 7, the device of claim 1, wherein the first model has lower computational requirements and/or a lower storage size requirement than the second model (NPL section I “end devices such as embedded sensor nodes often have limited memory and battery budgets”, which “makes it an issue to fit models on the devices that meet the required accuracy and energy constraints”, wherein it can be reasonably interpreted that the first model stored in the edge device has lower computational requirements and/or a lower storage size requirement than the second model).
Regarding claim 8, the device of claim 1, wherein the first model comprises fewer convolutional layers than the second model (NPL Section I “An example of one such distributed approach is to combine a small NN model (less number of parameters) on end devices and a larger NN model (more number of parameters) in the cloud”, which can be interpreted both as fewer layers or layers with a smaller depth; Figures 3 and 4).
Regarding claim 9, the device of claim 1, wherein the learnable control function is configured to form the output based on features extracted from the data sample (NPL section I “a system could train a single end-to-end model, such as a DNN, and partition it between end devices and the cloud”).
Regarding claim 10, the device of claim 1, wherein the learnable control function is configured to be optimized in dependence on a series of data samples and their respective true outputs associated with the series of data samples (NPL section I “A joint training method that minimizes communication and resource usage for devices and maximizes usefulness of extracted features which are utilized in the cloud, while allowing low-latency classification via early exit for a high percentage of input samples”).
Regarding claim 11, the device of claim 10, wherein the first model and the second model are learnable models, each of the first and second models being configured to be optimized in dependence on the series of data samples and their respective true outputs associated with the series of data samples (NPL section III.A: “DDNN relies on a jointly trained DNN framework at all parts in the neural network, for both training and inference”; section III.C: “Let be y a one-hot ground-truth label vector, be an input sample x C and be the set of all possible labels. For each exit, the softmax cross entropy objective function can be written as (...) “).
Regarding claim 12, the device of claim 1, wherein the remote location is a cloud server (NPL section III.A: “For more difficult cases, the intermediate DNN output (up to the local exit) is sent to the cloud”).
Regarding claim 13, the device of claim 1, wherein the learnable control function is a neural network comprising one or more convolutional layers (NPL Section I “An example of one such distributed approach is to combine a small NN model (less number of parameters) on end devices and a larger NN model (more number of parameters) in the cloud”, which can be interpreted both as fewer layers or layers with a smaller depth; Figures 3 and 4).
Regarding claim 14, the device of claim 1, wherein the input data sample is an image or a time series of data (NPL section IV.B: “This dataset consists of images acquired at the same time from six cameras placed at different locations facing the same general area”).
Regarding claim 15, the device of claim 1, wherein the device is a network node or an edge device in a communications network (NPL section II.A: “must be processed locally at the devices or at the edge, for otherwise the total amount of sensor data for a centralized cloud would overwhelm the communication network bandwidth”).
Regarding claim 16, this claim contains limitations found within those of claim 1, and the same rationale of rejection applies, where applicable.
Regarding claim 17, this claim contains limitations found within those of claim 1, and the same rationale of rejection applies, where applicable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: McAfoose et al US 2023/0031052.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT RASHID whose telephone number is (571)272-5372. The examiner can normally be reached 10AM-6PM EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tonia L Dollinger can be reached at 571-272-4170. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.R/Examiner, Art Unit 2459 /SCHQUITA D GOODWIN/Primary Examiner, Art Unit 2459