DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The IDS(s) has/have been considered and placed in the application file.
Election/Restrictions
Claim 8-10 are rejoined and the restriction requirement is withdrawn. Pursuant to 37 CFR 1.104 and MPEP § 821.04(B), claims 8-10 are rejected on the same art of record as claim 1 and 7, as the prior art search conducted on the elected claims fully encompasses the subject matter of the nonelected claims, and rejecting these claims now avoids unnecessary expenditure of time and resources by both the Office and Applicant.
Because all claims previously withdrawn from consideration under 37 CFR 1.142 have been rejoined, the restriction requirement as set forth in the Office action mailed on January 9, 2026, is hereby withdrawn. In view of the withdrawal of the restriction requirement as to the rejoined inventions, applicant(s) are advised that if any claim presented in a divisional application is anticipated by, or includes all the limitations of, a claim that is allowable in the present application, such claim may be subject to provisional statutory and/or nonstatutory double patenting rejections over the claims of the instant application. Once the restriction requirement is withdrawn, the provisions of 35 U.S.C. 121 are no longer applicable. See In re Ziegler, 443 F.2d 1211, 1215, 170 USPQ 129, 131-32 (CCPA 1971). See also MPEP § 804.01.
35 USC § 101 Analysis, Claims 1 and 7
Claims 1-10 are directed to statutory subject matter under 35 U.S.C. § 101. While the claims recite operations involving feature extraction and neural network inference implicating mathematical concepts, the claims as a whole integrate any such judicial exceptions into a practical application under Step 2A, Prong Two. The specification sets forth a concrete technical improvement in distributed video processing – specifically, portioning a single neural network across physically distinct hardware tiers and transmitting intermediate inference results between them, with segmentation points dynamically adjusted based on real-time device load status (see [0043]-[0046]). The claims reflect this improvement through the specific recitation of the three-tier sequential pipeline, the front-end extractions of universal features from video data, and the transmission of a first and second intermediate results between tiers (see claims 1-4 and 7-10). Accordingly, no rejection under 35 U.S.C. § 101 is warranted.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4 and 6-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liang (CN 112348172 A, hereinafter "Liang," published 2021-02-09, machine translation relied upon) in view of Jia (CN 111090773 A, hereinafter "Jia," published 2020-05-01, machine translation relied upon).
Note: Jia (CN 111090773 A) lists Peking University as assignee. This is the same entity as
the Applicant. The examiner considered whether the §102(b)(1)(A) exception applied given the same assignee relationship and the sub-one-year gap, but determined that the inventive entities are not the same and therefore the exception does not apply.
Claim 1.
The combination of Liang and Jia discloses an end-edge-cloud coordination system (Liang, “The invention relates to a deep neural network collaborative reasoning method based on an edge cloud framework, and the reasoning of the deep neural network is divided into an asynchronous model segmentation point evaluation stage and a real-time edge cloud collaborative reasoning stage … the neural network is divided into 3 parts according to the 2 segmentation points, wherein the parts are marked as P1, P2 and P3. The model computing task of the P1 segment is executed at the end side, the model computing task of the P2 segment is executed at the edge side, and the model computing task of the P3 segment is executed at the cloud side” p. 4. Liang discloses a terminal side, an edge side, and a cloud side that collaborate in a deep neural network inference pipeline) based on a digital retina, wherein the end-edge-cloud coordination system comprises a front-end device, an edge device and a cloud device (Liang, “the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology” p. 4. Three tiers – front end device, edge device, cloud device.);
the front-end device is configured to extract (Liang, “6) And the end side calculates all inference subtasks before the first segmentation point of the deep neural network according to the first segmentation point, and sends an inference intermediate result to the side” p. 10. “step S6.1: and the terminal calculates an inference task before the first segmentation point” p. 6. The front-end processes tasks, obtains first intermediate result, and sends to edge.);
the edge device is configured to process the analysis and recognition tasks based on the first intermediate result to obtain a second intermediate result to be sent to the cloud device (Liang
“7) And the side takes the inference intermediate result sent from the end side as input, calculates an inference subtask from a first segmentation point to a second segmentation point according to neural network layering, and sends the output intermediate result to the
cloud side” p. 10. “Step S6.2: and the terminal side sends the intermediate result calculated in the step S6.1 to the edge side, the edge side directly imports the intermediate result of the terminal side into the corresponding first segmentation point, calculates a calculation task from the first segmentation point to the second segmentation point, and sends the intermediate result of the edge side to the cloud side” p. 6. The edge processes base on the first intermediate result, obtains a second intermediate result, and sends it to the cloud.); and
the cloud device is configured to process the analysis and recognition tasks based on the second intermediate result to generate an analysis and recognition result of video data (Liang “8) And the cloud side takes the inference intermediate result sent from the side as input, calculates the inference subtask from the second division point to the last layer according to the hierarchy of the neural network, and obtains the final inference result” p. 10. “Step S6.3: and the cloud side uses the same reasoning framework to import the received intermediate result of the edge side in the step S6.2 into the second segmentation point, and the computing task after the second segmentation point is computed is completed” p. 6. The cloud processes based on the second intermediate results and generates the final result.).
Liang discloses all of the subject matter as described above except for specifically teaching “features with universality from collected video data and generate analysis and recognition tasks based on the features.” However, Jia in the same field of endeavor teaches “features with universality from collected video data and generate analysis and recognition tasks based on the features” (Jia “processing and converting the video stream into a video concentrated stream and a characteristic stream at front-end monitoring equipment … compressing the video stream to obtain a video concentrated stream; carrying out input adaptation and system scheduling on the video stream, and then carrying out preprocessing … A digital retina architecture and software architecture system, comprising: a front-end monitoring device, a cloud server and a terminal, the front-end monitoring equipment is used for converting the video stream into a video concentrated stream and a characteristic stream in the front-end monitoring equipment, receiving target characteristics sent by the cloud server, and sending a matching result to the cloud server after real-time matching” pp. 2-3. Jia’s produces a single feature stream that serves offline retrieval, real-time tracking, and target matching i.e. features usable across multiple tasks “universality.”)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the terminal device of Liang to incorporate Jia's video feature extraction and task generation at the front-end. The motivation for this combination would have been to reduce the volume of data transmitted from the front-end to the edge by extracting and transmitting only universally relevant features rather than raw video, thereby reducing bandwidth consumption and enabling more efficient distributed analysis across the end-edge-cloud pipeline.
Claim 2.
The combination of Liang and Jia discloses the end-edge-cloud coordination system based on the digital retina according to claim 1, wherein the front-end device is configured to process the analysis and recognition tasks by using a first number of target layers in a neural network model trained by the cloud device (Liang discloses the model is trained and distributed from the cloud (Step S1): “The deep neural network model is obtained through training based on the following and not limited to deep learning frames TensorFlow and Pytorch, and the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology. The neural network model can also be trained in advance and stored in a model warehouse of the cloud, and the terminal, the edge and the corresponding cloud download the model of the same version from the same position” p. 4. Liang also teaches the front-end using the layers before the fist split point (Steps S4/S6): “The model computing task of the P1 segment is executed at the end side” p. 4. Liang further teaches, “step S6.1: and the terminal calculates an inference task before the first segmentation point” p. 6.).
Claim 3.
The combination of Liang and Jia discloses the end-edge-cloud coordination system based on the digital retina according to claim 2, wherein the edge device is configured to process the analysis and recognition tasks by using a second number of target layers in the neural network model trained by the cloud device (Liang disclose the edge computing the layers between the two split points is the “second number of target layers,” “Step S6.2: and the terminal side sends the intermediate result calculated in the step S6.1 to the edge side, the edge side directly imports the intermediate result of the terminal side into the corresponding first segmentation point, calculates a calculation task from the first segmentation point to the second segmentation point, and sends the intermediate result of the edge side to the cloud side” p. 6.).
Claim 4.
The combination of Liang and Jia discloses the end-edge-cloud coordination system based on the digital retina according to claim 3, wherein the cloud device is configured to process the analysis and recognition tasks by using a third number of target layers in the neural network model trained by the cloud device; and wherein the neural network model comprises the first number of target layers, the second number of target layers and the third number of target layers connected in sequence (Liang step S6.3 and the model description “the cloud side uses the same reasoning framework to import the received intermediate result of the edge side in the step S6.2 into the second segmentation point, and the computing task after the second segmentation point is computed is completed” p. 6. And for the layers-in-sequence piece, “the neural network is divided into 3 parts according to the 2 segmentation points, wherein the parts are marked as P1, P2 and P3. The model computing task of the P1 segment is executed at the end side, the model computing task of the P2 segment is executed at the edge side, and the model computing task of the P3 segment is executed at the cloud side” p. 4. The first quote covers the cloud’s third segment; the second quote shows all three segments connected in sequence across the three tiers.).
Claim 6.
The combination of Liang and Jia discloses the end-edge-cloud coordination system based on the digital retina according to claim 1, wherein: the front-end device is a video capture device; the edge device is an edge server; and the cloud device is a cloud server (Jia teaches that the front-end device is a video acquisition/monitoring device “A digital retina architecture and software architecture system, comprising: a front-end monitoring device, a cloud server and a terminal, the front-end monitoring equipment is used for converting the video stream into a video concentrated stream and a characteristic stream” p. 3. Liang teaches that the edge side and cloud side are server-type computing devices. It would have been obvious that the edge device is an edge server and the cloud device is a cloud server, as these are the standard computing devices used at the edge and cloud tiers in distributed computing architectures.)
Claim 7.
Claim 7 is a method claim reciting substantially similar steps to the system of claim 1. Claim 7 is rejected under the combination of Liang and Jia for same rationale as claim 1 above.
Claim 8.
The combination of Liang and Jia discloses a front-end device (Liang, “the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology” p. 4. Three tiers – front end device, edge device, cloud device.), wherein the front-end device comprises a camera, a memory and one or more processors (Liang “the deep neural network model is obtained through training … and the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology … The end-side devices include, but are not limited to, cell phones, raspberry pies, drones, end-side development board Jetson series” pp. 4-5. Liang Step S1 discloses the terminal-side hardware with on-device model and inference capability.); the camera is configured to collect video data; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the following steps (Jia “A digital retina architecture and software architecture system, comprising: a front-end monitoring device, a cloud server and a terminal, the front-end monitoring equipment is used for converting the video stream into a video concentrated stream and a characteristic stream in the front-end monitoring equipment, receiving target characteristics sent by the cloud server, and sending a matching result to the cloud server” p. 3. Jia teaches a front-end monitor device as a video capture apparatus.) …
The functional steps mirror claim 1’s limitations and are addressed by Liang and Jia under the same rationale.
Claim 9.
The combination of Liang and Jia discloses an edge device, wherein the edge device comprises a memory and one or more processors (Liang, “the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology” p. 4. Three tiers – front end device, edge device, cloud device.); a computer program is stored in the memory, and when the computer program is executed by the processor (Liang “in the memory of the side” p. 2. Liang “a CPU, a GPU” p. 3. Liang “the same inference engine is needed for the terminal, the edge end and the cloud end” p. 6. An inference engine is a software program the runs on each device’s processor.), the processor executes the following steps …
The functional steps mirror claim 1’s limitations and are addressed by Liang and Jia under the same rationale.
Claim 10.
The combination of Liang and Jia discloses a cloud device (Liang, “the same deep neural network model is downloaded to a terminal side, an edge side and a cloud side through a data synchronization technology” p. 4. Three tiers – front end device, edge device, cloud device.), wherein the cloud device comprises a memory and one or more processors; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the following steps (Liang “in the memory of the side” p. 2. Liang “a CPU, a GPU” p. 3. Liang “the same inference engine is needed for the terminal, the edge end and the cloud end” p. 6. An inference engine is a software program the runs on each device’s processor.) …
The functional steps mirror claim 1’s limitations and are addressed by Liang and Jia under the same rationale.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Liang in view of Jia and further in view of Alamouti et al. (US 2020/0322225 A1, hereinafter "Mimik").
Claim 5.
The combination of Liang and Jia discloses the end-edge-cloud coordination system based on the digital retina according to claim 1, wherein (Liang and Jia teach the end-edge-cloud coordination system as applied to claim 1 above):
Liang and Jia discloses all of the subject matter as described above except for specifically teaching “a plurality of the front-end devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween; a plurality of the edge devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween; and a plurality of the cloud devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween.” However, Mimik in the same field of endeavor teaches a plurality of the front-end devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween; a plurality of the edge devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween (Mimik teaches same device discovery, cluster formation, and collaborative resource/data sharing “dynamically form one or more clusters with the one or more edge cloud computing devices; and communicate with the one or more edge cloud computing devices at a microservice level either directly or through other edge cloud computing devices across the one or more clusters,” claim 4.); and a plurality of the cloud devices are used for allocation and data exchange of the same level of analysis and recognition tasks therebetween (Mimik “connect with the discovered one or more edge cloud computing devices when the discovered one or more edge cloud computing devices chose to share data, services, and/or resources,” claim 5.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the end-edge-cloud system of Liang/Jia to incorporate Mimik's same-level device collaboration and resource sharing at each tier. The motivation for this combination of references would have been to improve system scalability and fault tolerance by enabling load balancing and redundancy among multiple devices at each level of the architecture, thereby allowing the system to handle varying computational demands and maintain service continuity when individual devices are overloaded or unavailable.
Conclusion
The prior art made of record but not relied, yet considered pertinent to the applicant’s disclosure, is listed on the PTO-892 form.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights