DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 21-40 are pending this office action.
Response to Amendment
This Office Action is in response to applicant’s communication filed on June 3rd, 2026. The applicant’s remark and amendments to the claims were considered with the results that follow.
In response to the last Office Action, claims 21, 26, 28-31, 33-34, and 39 have been amended. No claims are canceled or added. As a result, claims 21-40 are pending in this application.
Response to Arguments
Applicant’s arguments with respect to claims 21, 28, and 34 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21, 28, and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature "LAVEA: Latency-aware Video Analytics on Edge Computing Platform" issued to Yi et al. (hereinafter as "Yi") in view of Non-Patent Literature “DARE: Dynamic Adaptive Mobile Augmented Reality with Edge Computing issued to Liu et al. (hereinafter as “Liu”) in further view of Non-Patent Literature “Edge Enhanced Deep Learning Systems for Large-scale Video Stream Analytics” issued to Ali et al. (hereinafter as “Ali”).
Regarding claim 21, Yi teaches a system comprising: a processor (Yi: Abstract, pg. 1; In this paper, we present LAVEA, a system built on top of an edge computing platform, which offloads computation between clients and edge nodes, collaborates nearby edge nodes, to provide. 4.1 Task O loading System Model and Problem Formulation, pg. 5-6; where pi is the processor speed of client i. where p0 is the processor speed of the edge server. 6.2 Evaluation Setup, pg. 8; These four machines have the same hardware specifications. They all have a quad-core CPU and 4 GB main memory); and a storage memory storing computer-readable instructions, which when executed by the processor (Yi: 6.1 Implementation Details, pg. 8; The task queue is implemented using Redis as it is in memory and has very good performance. 6.1 Implementation Details, pg. 8; Our implementation aims at a serverless edge computing architecture. As shown in system architecture of Fig. 4, our implementation is based on docker container…creation and destruction of docker instances is much faster than that of VM instances. Inspired by the IBM OpenWhisk [18], each worker container contains an action proxy, which uses Python to run any scripts or compile and execute any binary executable. The end user only needs to 1) deploy our edge computing platform on heterogeneous devices with just a click, 2) define the event of interests using a provided API, and 3) provide a function (scripts or binary executable) to process such event. 6.2 Evaluation Setup, pg. 8; These four machines have the same hardware specifications. They all have a quad-core CPU and 4 GB main memory), cause the processor to: receive a video query regarding a live video stream recorded by at least one camera (Yi: 1 INTRODUCTION, pg. 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. For example, in “Amber Alert”, our system can automate and speedup the searching of objects of interest by vehicle recognition, vehicle license plate recognition and face recognition utilizing various web cameras deployed at highway entrances, or dash cameras or cameras of smartphones mounted on cars. 4.1 Task O loading System Model and Problem Formulation, pg. 6; Each client i, I ∈[1,N], will process the upcoming job upon request, e.g. recognizing the license plates in video streams);
determine a resource requirement estimate for the video query (Yi: 3.3 Edge Computing Services, 3.3.1 Profiler Service, pg. 4; we have found that the execution of video process tasks is relatively stable (when input and algorithmic configurations are given) and a profiler can be used to collect relevant metrics. Therefore, we add a profiling phase to the deployment of every new type of client devices and edge devices. The profiler will execute instrumented tasks multiple times with different inputs and configurations on the device and measure metrics including but not limited to execution time, input/output data size, etc);
determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query (Yi: Abstract, pg. 1; we have designed and compared various task placement schemes that are tailed for inter-edge collaboration. Our results reveal that the client-edge configuration has a speedup ranging from 1.3x to 4x (1.2x to 1.7x) against running in local (client-cloud configuration). Section 1 INTRODUCTION, pg. 1; We have evaluated several task placement schemes for inter-edge collaboration and proposed a predication-based method which efficiently estimates the response time. 4.1 Task Offloading System Model and Problem Formulation, pg.5; Thus, our offloading problem can be taken as a graph partition problem, in which we need to assign a directed graph of tasks to different computing nodes (local, edge, or cloud), with the purpose to minimize certain cost);
determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment (Yi: 2.1 Edge Computing Network, pg. 2-3; When a client connects to the edge node, we implicitly indicate that the client will first connect to the correspond access points (APs) using cable or wireless and then utilize the services provided by the co-located edge node…a client will only connect to one of the available edge nodes nearby at certain location. 3.2 System Overview: 3.2.1 Edge Computing Node, pg. 4; In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.3 Edge Computing Services: 3.3.2 Monitoring Service, pg. 4; Unlike profiler service which gathers pre-run-time execution information on pre-defined inputs and configurations, the monitoring service is used to continuously monitor and collect run-time information such as the network, system load, etc., from not only the clients but also nearby edge nodes. 3.3.3 Offloading Service, pg. 4. The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 5; Unlike existing offloading frameworks for mobile cloud computing, we take the resource contention or scheduling delay at the edge side into consideration by adding an end-to-end delay constraint);
allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration (Yi: ABSTRACT, pg 1; In this paper, we present LAVEA, a system built on top of an edge computing platform, which offloads computation between clients and edge nodes, collaborates nearby edge nodes, to provide low-latency video analytics at places closer to the users. Section 1. Introduction, pg 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. 3.2.1 Edge Computing Node, pg. 4; In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.3.3 Offloading Service, pg. 4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Thus, our offloading problem can be taken as a graph partition problem, in which we need to assign a directed graph of tasks to different computing nodes (local, edge, or cloud), with the purpose to minimize certain cost);
Yi does not explicitly teach detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition.
However, Liu teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition (Liu: Abstract, pg. 1; In this paper, we design a new network protocol named DARE (dynamic adaptive AR over the edge) that enables mobile users to dynamically change their AR configurations according to wireless channel conditions and computation workloads in edge servers. I. INTRODUCTION, pg. 1; we design the DARE (dynamic adaptive AR over the edge) protocol that enables the edge-based MAR system to dynamically change AR configurations and computation resource allocations according to wireless channel conditions and available computation resources on the edge server. VI. PERFORMANCE EVALUATION, A. Protocol Implementation, pg. 7-8; The QoA monitor will trigger the optimization request to the edge server if the wireless channel condition changes or the service latency exceeds Tmax. C. Evaluation Results, pg. 10; Video framerate and pipelining: With the DARE protocol, the MAR client dynamically changes the AR framerate according to workloads on the edge server. Fig.14 shows the impact of the video frame rate on the server workload);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving the accuracy and performance of the edge computing system by exploiting collaboration to reduce the latency (See Liu: II. RELATED WORK, Mobile Edge Computing). In addition, the references (Yi and Liu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi and Liu are directed to video analytic system based on offloading and awareness.
The modification of Yi and Liu teaches claimed invention substantially as claimed, however, the modification of Yi and Liu does not explicitly teach when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation.
However, Ali teaches when the detected change in resource availability is insufficient for the initial local processing of the video query (Ali: Abstract, pg. 1; The proposed approach performs initial processing of the data close to the data source at edge and fog nodes, resulting in significant reduction in the data that is transferred and stored in the cloud. IV. EDGE ENHANCED DEEP LEARNING MODEL, pg. 4; In general, we perform non-compute intensive stages on the edge tier and more compute-intensive stages on the in-transit and cloud tiers. For S1 and S2, edge tier can be used. Edge tier is defined as consisting of the data sources (cameras) and the edge nodes. For S3 machine learning model to detect the object is run on the cloudlet node. For S4, the trained deep learning model is run on the cloud to perform the compute-intensive work of object recognition. V. EXPERIMENTAL SETUP, pg. 6; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied),
reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation (Ali: Abstract, pg. 1; We address this problem by distributing the deep learning pipeline across edge and cloudlet/fog resources. The basic processing stages and trained models are distributed towards the edge of the network and on in-transit and cloud resources. V. EXPERIMENTAL SETUP, pg. 6-7; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied. I. If a network node is unable to process the job under its respective deadline time, the job is forwarded to the next hop in the network or rejected in case of a cloud).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the further teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving performance of the video analytic system while keeping the accuracy intact (See Ali: I. INTRODUCTION). In addition, the references (Yi, Liu, and Ali) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, and Ali are directed to video analytic system based on offloading and awareness.
Regarding claim 28, Yi teaches a method, comprising: determining a configuration for performing local processing of input data (Yi: Section 3.2.2 Edge Client, pg. 4. Since most edge clients are either resource constrained devices or need to accommodate requests from a large number of clients, an edge client usually runs lightweight data processing tasks locally and offloads heavy tasks to the edge computing node nearby. 3.3 Edge Computing Services: 3.3.3 Offloading Service, pg.4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4 EDGE-FRONT OFFLOADING, pg. 5; In this section, we consider selecting tasks to run on the edge as a computation offloading problem), received from a first device (Yi: Section 1. Introduction, pg. 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. 3.2 System Overview: 3.2.2 Edge Client, pg. 4; In LAVEA, the edge client has a thin client design, to make sure all the clients can run it without introducing too much overhead. 4.1 Task O loading System Model and Problem Formulation, pg. 6; Each client i, I ∈[1,N], will process the upcoming job upon request, e.g. recognizing the license plates in video streams), based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location with an edge computing environment (Yi: 2.1 Edge Computing Network, pg. 2-3; When a client connects to the edge node, we implicitly indicate that the client will first connect to the correspond access points (APs) using cable or wireless and then utilize the services provided by the co-located edge node…a client will only connect to one of the available edge nodes nearby at certain location. 3.2.1 Edge Computing Node, pg. 4. In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.3 Edge Computing Services; 3.3.2 Monitoring Service, pg. 4; Unlike profiler service which gathers pre-run-time execution information on pre-defined inputs and configurations, the monitoring service is used to continuously monitor and collect run-time information such as the network, system load, etc., from not only the clients but also nearby edge nodes. 3.3.3 Offloading Service, pg. 4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Unlike existing offloading frameworks for mobile cloud computing, we take the resource contention or scheduling delay at the edge side into consideration by adding an end-to-end delay constraint);
allocating initial local processing of the input datathe at least one edge device and one or more cameras according to the determined configuration (Yi: ABSTRACT, pg. 1; In this paper, we present LAVEA, a system built on top of an edge computing platform, which offloads computation between clients and edge nodes, collaborates nearby edge nodes, to provide low-latency video analytics at places closer to the users. Section 1. Introduction, pg. 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. 3.2 System Overview: 3.2.1 Edge Computing Node, pg. 4; In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.2 System Overview: 3.3.3 Offloading Service, pg. 4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Thus, our offloading problem can be taken as a graph partition problem, in which we need to assign a directed graph of tasks to different computing nodes (local, edge, or cloud), with the purpose to minimize certain cost);
Yi does not explicitly teach detecting a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition.
However, Liu teaches detecting a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition (Liu: Abstract, pg. 1; In this paper, we design a new network protocol named DARE (dynamic adaptive AR over the edge) that enables mobile users to dynamically change their AR configurations according to wireless channel conditions and computation workloads in edge servers. I. INTRODUCTION, pg. 1; we design the DARE (dynamic adaptive AR over the edge) protocol that enables the edge-based MAR system to dynamically change AR configurations and computation resource allocations according to wireless channel conditions and available computation resources on the edge server. VI. PERFORMANCE EVALUATION, A. Protocol Implementation, pg. 7-8; The QoA monitor will trigger the optimization request to the edge server if the wireless channel condition changes or the service latency exceeds Tmax. C. Evaluation Results, pg. 10; Video framerate and pipelining: With the DARE protocol, the MAR client dynamically changes the AR framerate according to workloads on the edge server. Fig.14 shows the impact of the video frame rate on the server workload);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving the accuracy and performance of the edge computing system by exploiting collaboration to reduce the latency (See Liu: II. RELATED WORK, Mobile Edge Computing). In addition, the references (Yi and Liu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi and Liu are directed to video analytic system based on offloading and awareness.
The modification of Yi and Liu teaches claimed invention substantially as claimed, however, the modification of Yi and Liu does not explicitly teach when the detected change in resource availability is insufficient for the initial local processing of the input data, reallocating to one or more cloud devices
However, Ali teaches when the detected change in resource availability is insufficient for the initial local processing of the input data (Ali: Abstract, pg. 1; The proposed approach performs initial processing of the data close to the data source at edge and fog nodes, resulting in significant reduction in the data that is transferred and stored in the cloud. IV. EDGE ENHANCED DEEP LEARNING MODEL, pg. 4; In general, we perform non-compute intensive stages on the edge tier and more compute-intensive stages on the in-transit and cloud tiers. For S1 and S2, edge tier can be used. Edge tier is defined as consisting of the data sources (cameras) and the edge nodes. For S3 machine learning model to detect the object is run on the cloudlet node. For S4, the trained deep learning model is run on the cloud to perform the compute-intensive work of object recognition. V. EXPERIMENTAL SETUP, pg. 6; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied),
reallocating to one or more cloud devices We address this problem by distributing the deep learning pipeline across edge and cloudlet/fog resources. The basic processing stages and trained models are distributed towards the edge of the network and on in-transit and cloud resources. V. EXPERIMENTAL SETUP, pg. 6-7; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied. I. If a network node is unable to process the job under its respective deadline time, the job is forwarded to the next hop in the network or rejected in case of a cloud).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the further teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving performance of the video analytic system while keeping the accuracy intact (See Ali: I. INTRODUCTION). In addition, the references (Yi, Liu, and Ali) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, and Ali are directed to video analytic system based on offloading and awareness.
Regarding claim 34, Yi teaches a method, comprising:receiving a video query regarding a live video stream recorded by at least one camera (Yi: 1 INTRODUCTION, pg. 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. For example, in “Amber Alert”, our system can automate and speedup the searching of objects of interest by vehicle recognition, vehicle license plate recognition and face recognition utilizing various web cameras deployed at highway entrances, or dash cameras or cameras of smartphones mounted on cars. 4.1 Task O loading System Model and Problem Formulation, pg. 6; Each client i, I ∈[1,N], will process the upcoming job upon request, e.g. recognizing the license plates in video streams);
determining a resource requirement estimate for the video query (Yi: 3.3 Edge Computing Services, 3.3.1 Profiler Service, pg. 4; we have found that the execution of video process tasks is relatively stable (when input and algorithmic configurations are given) and a profiler can be used to collect relevant metrics. Therefore, we add a profiling phase to the deployment of every new type of client devices and edge devices. The profiler will execute instrumented tasks multiple times with different inputs and configurations on the device and measure metrics including but not limited to execution time, input/output data size, etc);
determining a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query (Yi: Abstract, pg. 1; we have designed and compared various task placement schemes that are tailed for inter-edge collaboration. Our results reveal that the client-edge configuration has a speedup ranging from 1.3x to 4x (1.2x to 1.7x) against running in local (client-cloud configuration). Section 1 INTRODUCTION, pg. 1; We have evaluated several task placement schemes for inter-edge collaboration and proposed a predication-based method which efficiently estimates the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Thus, our offloading problem can be taken as a graph partition problem, in which we need to assign a directed graph of tasks to different computing nodes (local, edge, or cloud), with the purpose to minimize certain cost);
determining a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment (Yi: 2.1 Edge Computing Network, pg. 2-3; When a client connects to the edge node, we implicitly indicate that the client will first connect to the correspond access points (APs) using cable or wireless and then utilize the services provided by the co-located edge node…a client will only connect to one of the available edge nodes nearby at certain location. 3.2 System Overview: 3.2.1 Edge Computing Node, pg. 4. In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.3 Edge Computing Services; 3.3.3 Offloading Service, pg. 4. The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 3.3 Edge Computing Services; 3.3.2 Monitoring Service, pg. 4. Unlike profiler service which gathers pre-run-time execution information on pre-defined inputs and configurations, the monitoring service is used to continuously monitor and collect run-time information such as the network, system load, etc., from not only the clients but also nearby edge nodes. 3.3 Edge Computing Services: 3.3.3 Offloading Service, pg. 4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Unlike existing offloading frameworks for mobile cloud computing, we take the resource contention or scheduling delay at the edge side into consideration by adding an end-to-end delay constraint);
allocate initial local processing for the video query between the at least one edge device according to the determined configuration (Yi: ABSTRACT, pg. 1; In this paper, we present LAVEA, a system built on top of an edge computing platform, which offloads computation between clients and edge nodes, collaborates nearby edge nodes, to provide low-latency video analytics at places closer to the users. Section 1. Introduction, pg. 1; The ability to provide low latency video analytics is critical for applications in the elds of public safety, counter-terrorism, self-driving cars, VR/AR, etc [32]. In video edge analytic applications, we consider typical client devices such as mobile phones, body-worn cameras or dash cameras mounted on vehicles, web cameras at toll stations or highway check points, security cameras in public places, or even video captured by UAVs [35]. 3.2 System Overview: 3.2.1 Edge Computing Node, pg. 4. In LAVEA, the edge computing node provides edge computing services to the mobile devices nearby, the edge computing node attached to the same access point or base station as clients is called the edge-front. 3.2 System Overview: 3.3.3 Offloading Service, pg. 4; The offloading controller will track tasks running locally at the client, and exchange information with the offloading service running on the edge-front server. The variables gathered in profiler and monitoring services will be used as inputs to the offloading decision problem which is formulated as an optimization problem to minimize the response time. 4.1 Task Offloading System Model and Problem Formulation, pg. 6; Thus, our offloading problem can be taken as a graph partition problem, in which we need to assign a directed graph of tasks to different computing nodes (local, edge, or cloud), with the purpose to minimize certain cost);
Yi does not explicitly teach detecting a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition.
However, Liu teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition (Liu: Abstract, pg. 1; In this paper, we design a new network protocol named DARE (dynamic adaptive AR over the edge) that enables mobile users to dynamically change their AR configurations according to wireless channel conditions and computation workloads in edge servers. I. INTRODUCTION, pg. 1; we design the DARE (dynamic adaptive AR over the edge) protocol that enables the edge-based MAR system to dynamically change AR configurations and computation resource allocations according to wireless channel conditions and available computation resources on the edge server. VI. PERFORMANCE EVALUATION, A. Protocol Implementation, pg. 7-8; The QoA monitor will trigger the optimization request to the edge server if the wireless channel condition changes or the service latency exceeds Tmax. C. Evaluation Results, pg. 10; Video framerate and pipelining: With the DARE protocol, the MAR client dynamically changes the AR framerate according to workloads on the edge server. Fig.14 shows the impact of the video frame rate on the server workload);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving the accuracy and performance of the edge computing system by exploiting collaboration to reduce the latency (See Liu: II. RELATED WORK, Mobile Edge Computing). In addition, the references (Yi and Liu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi and Liu are directed to video analytic system based on offloading and awareness.
The modification of Yi and Liu teaches claimed invention substantially as claimed, however, the modification of Yi and Liu does not explicitly teach when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation.
However, Ali teaches when the detected change in resource availability is insufficient for the initial local processing of the video query (Ali: Abstract, pg. 1; The proposed approach performs initial processing of the data close to the data source at edge and fog nodes, resulting in significant reduction in the data that is transferred and stored in the cloud. IV. EDGE ENHANCED DEEP LEARNING MODEL, pg. 4; In general, we perform non-compute intensive stages on the edge tier and more compute-intensive stages on the in-transit and cloud tiers. For S1 and S2, edge tier can be used. Edge tier is defined as consisting of the data sources (cameras) and the edge nodes. For S3 machine learning model to detect the object is run on the cloudlet node. For S4, the trained deep learning model is run on the cloud to perform the compute-intensive work of object recognition. V. EXPERIMENTAL SETUP, pg. 6; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied),
reallocate to one or more cloud devices We address this problem by distributing the deep learning pipeline across edge and cloudlet/fog resources. The basic processing stages and trained models are distributed towards the edge of the network and on in-transit and cloud resources. V. EXPERIMENTAL SETUP, pg. 6-7; The edge and in-transit nodes have limited memory storage space. When the memory buffer of an edge or in-transit resource is exceeded, it sends all the jobs in its current memory buffer to the next network hop and frees the space occupied. I. If a network node is unable to process the job under its respective deadline time, the job is forwarded to the next hop in the network or rejected in case of a cloud).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the further teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation). One of ordinary skill in the art would have been motivated to make such a combination is by providing better results in improving performance of the video analytic system while keeping the accuracy intact (See Ali: I. INTRODUCTION). In addition, the references (Yi, Liu, and Ali) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, and Ali are directed to video analytic system based on offloading and awareness.
Claims 22-24, 31-32, and 35-37 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature "LAVEA: Latency-aware Video Analytics on Edge Computing Platform" issued to Yi et al. (hereinafter as "Yi") in view of Non-Patent Literature “DARE: Dynamic Adaptive Mobile Augmented Reality with Edge Computing issued to Liu et al. (hereinafter as “Liu”) in view of Non-Patent Literature “Edge Enhanced Deep Learning Systems for Large-scale Video Stream Analytics” issued to Ali et al. (hereinafter as “Ali”) in further view of Non-Patent Literature "A Computing Platform for Video Crowdprocessing Using Deep Learning" issued to Lu et al. (hereinafter as "Lu").
Regarding claim 22, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however the modification of Yi, Liu, and Ali does not explicitly teach determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module.
Lu teaches the determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module (
PNG
media_image1.png
259
502
media_image1.png
Greyscale
See Lu pg 1431, III. OVERVIEW: We consider a crowdprocessing approach to perform object detection/classification of videos. This task includes filtering of videos based on metadata, and then processing the videos to perform object detection. Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. Alternatively, the user may perform frame extraction locally and then offload specific frames to the cloud. In this case, it needs to determine whether each frame is processed by (ii) frame offload in which the frame is sent to the cloud for detection or (iii) local detection where the frame is detected on the mobile device {Examiner correlates based on the frame extraction that heavy processing would be offloaded to the cloud to perform the video processing while low processing can be immediately by process the object detection then deliver to the cloud to perform the video processing}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by saving the energy usage by offloading to a different location to process video without the need of extra energy usage concern (See Lu: III. OVERVIEW; At the expense of some energy usage, the user can offload the videos to the cloud, where high performance computing can quickly process the videos without energy usage concerns). In addition, the references (Yi, Liu, Ali, and Lu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Lu are directed to video analytic system based on offloading and awareness.
Regarding claim 23, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, and Lu further teaches claimed invention substantially as claimed, and Lu further teaches the determined configuration directs the set of local devices to perform background subtraction on the extracted video frames (Lu: pg 1431, III. OVERVIEW: Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. pg.1432, A. Frame Extraction, “Frame extraction is used to take individual video frames and transform them into images upon which object detection may be performed. To target objects with different dynamics within the video, the task issuer may request a different frame extraction rate. For example, for an object moving at a high speed like a car, the rate should be high enough to not miss the object”. B. Detection, “However, batch processing performs much better, where the intercept (α) is about 240ms and the slope (β) is 400ms. The difference grows with the increase of the number of frames. For detection, it is better to put more frames in a batch to reduce processing time. However, the system must wait longer to get the extracted frames from the video. Therefore, it is difficult to determine the best batch size” {Examiner correlates the difference rate in detecting the object would determine the best batch size to be deliver}).
Regarding claim 24, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, and Lu further teaches the background subtraction is performed on the extracted video frames to determine whether additional processing should be performed (Lu: pg.1432, A. Frame Extraction, “Frame extraction is used to take individual video frames and transform them into images upon which object detection may be performed. To target objects with different dynamics within the video, the task issuer may request a different frame extraction rate. For example, for an object moving at a high speed like a car, the rate should be high enough to not miss the object”. B. Detection, “However, batch processing performs much better, where the intercept (α) is about 240ms and the slope (β) is 400ms. The difference grows with the increase of the number of frames. For detection, it is better to put more frames in a batch to reduce processing time. However, the system must wait longer to get the extracted frames from the video. Therefore, it is difficult to determine the best batch size” {Examiner correlates the additional processing based on observing the difference growth in the rate when detecting the object in which allows to put more frames in the batch in which requires additional processing to determine best batch size}).
Regarding claim 31, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however the modification of Yi, Liu, and Ali does not explicitly teach the allocated processing of input data directs the set of local devices to extract video frames from a live video stream using a decoding module.
Lu teaches the allocated processing of input data directs the set of local devices to extract video frames from the live video stream using a decoding module (
PNG
media_image1.png
259
502
media_image1.png
Greyscale
See Lu pg 1431, III. OVERVIEW: We consider a crowdprocessing approach to perform object detection/classification of videos. This task includes filtering of videos based on metadata, and then processing the videos to perform object detection. Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. Alternatively, the user may perform frame extraction locally and then offload specific frames to the cloud. In this case, it needs to determine whether each frame is processed by (ii) frame offload in which the frame is sent to the cloud for detection or (iii) local detection where the frame is detected on the mobile device {Examiner correlates based on the frame extraction that heavy processing would be offloaded to the cloud to perform the video processing while low processing can be immediately by process the object detection then deliver to the cloud to perform the video processing}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by saving the energy usage by offloading to a different location to process video without the need of extra energy usage concern (See Lu: III. OVERVIEW; At the expense of some energy usage, the user can offload the videos to the cloud, where high performance computing can quickly process the videos without energy usage concerns). In addition, the references (Yi, Liu, Ali, and Lu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Lu are directed to video analytic system based on offloading and awareness.
Regarding claim 32, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, and Lu further teaches the allocated processing of input data directs the set of local devices to perform background subtraction on the extracted video frames (Lu: pg 1431, III. OVERVIEW: Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. pg.1432, A. Frame Extraction, “Frame extraction is used to take individual video frames and transform them into images upon which object detection may be performed. To target objects with different dynamics within the video, the task issuer may request a different frame extraction rate. For example, for an object moving at a high speed like a car, the rate should be high enough to not miss the object”. B. Detection, “However, batch processing performs much better, where the intercept (α) is about 240ms and the slope (β) is 400ms. The difference grows with the increase of the number of frames. For detection, it is better to put more frames in a batch to reduce processing time. However, the system must wait longer to get the extracted frames from the video. Therefore, it is difficult to determine the best batch size” {Examiner correlates the difference rate in detecting the object would determine the best batch size to be deliver}).
Regarding claim 35, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however the modification of Yi, Liu, and Ali does not explicitly teach the determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module.
Lu teaches the determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module (
PNG
media_image1.png
259
502
media_image1.png
Greyscale
See Lu pg 1431, III. OVERVIEW: We consider a crowdprocessing approach to perform object detection/classification of videos. This task includes filtering of videos based on metadata, and then processing the videos to perform object detection. Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. Alternatively, the user may perform frame extraction locally and then offload specific frames to the cloud. In this case, it needs to determine whether each frame is processed by (ii) frame offload in which the frame is sent to the cloud for detection or (iii) local detection where the frame is detected on the mobile device {Examiner correlates based on the frame extraction that heavy processing would be offloaded to the cloud to perform the video processing while low processing can be immediately by process the object detection then deliver to the cloud to perform the video processing}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by saving the energy usage by offloading to a different location to process video without the need of extra energy usage concern (See Lu: III. OVERVIEW; At the expense of some energy usage, the user can offload the videos to the cloud, where high performance computing can quickly process the videos without energy usage concerns). In addition, the references (Yi, Liu, Ali, and Lu) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Lu are directed to video analytic system based on offloading and awareness.
Regarding claim 36, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, and Lu further teaches the determined configuration directs the set of local devices to perform background subtraction on the extracted video frames (Lu: pg 1431, III. OVERVIEW: Alternatively, the user can process some of the frames locally… Once these videos are processed using deep learning either locally on the mobile device…For frames that are processed on the mobile devices, the user will forward either the tags, the frames of interest. pg.1432, A. Frame Extraction, “Frame extraction is used to take individual video frames and transform them into images upon which object detection may be performed. To target objects with different dynamics within the video, the task issuer may request a different frame extraction rate. For example, for an object moving at a high speed like a car, the rate should be high enough to not miss the object”. B. Detection, “However, batch processing performs much better, where the intercept (α) is about 240ms and the slope (β) is 400ms. The difference grows with the increase of the number of frames. For detection, it is better to put more frames in a batch to reduce processing time. However, the system must wait longer to get the extracted frames from the video. Therefore, it is difficult to determine the best batch size” {Examiner correlates the difference rate in detecting the object would determine the best batch size to be deliver}).
Regarding claim 37, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, and Lu further teaches the background subtraction is performed on the extracted video frames to determine whether additional processing should be performed (Lu: pg.1432, A. Frame Extraction, “Frame extraction is used to take individual video frames and transform them into images upon which object detection may be performed. To target objects with different dynamics within the video, the task issuer may request a different frame extraction rate. For example, for an object moving at a high speed like a car, the rate should be high enough to not miss the object”. B. Detection, “However, batch processing performs much better, where the intercept (α) is about 240ms and the slope (β) is 400ms. The difference grows with the increase of the number of frames. For detection, it is better to put more frames in a batch to reduce processing time. However, the system must wait longer to get the extracted frames from the video. Therefore, it is difficult to determine the best batch size” {Examiner correlates the additional processing based on observing the difference growth in the rate when detecting the object in which allows to put more frames in the batch in which requires additional processing to determine best batch size}).
Claims 25-26, 33, and 38-39 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature "LAVEA: Latency-aware Video Analytics on Edge Computing Platform" issued to Yi et al. (hereinafter as "Yi") in view of Non-Patent Literature “DARE: Dynamic Adaptive Mobile Augmented Reality with Edge Computing issued to Liu et al. (hereinafter as “Liu”) in view of Non-Patent Literature “Edge Enhanced Deep Learning Systems for Large-scale Video Stream Analytics” issued to Ali et al. (hereinafter as “Ali”) in view of Non-Patent Literature "A Computing Platform for Video Crowdprocessing Using Deep Learning" issued to Lu et al. (hereinafter as "Lu") in further view of Non-Patent Literature "Delivering Deep Learning to Mobile Devices via Offloading" issued to Xukan Ran et al. (hereinafter as “Ran”).
Regarding claim 25, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, Ali, and Lu does not explicitly teach the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices.
Ran teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices (Ran, Introduction pg. 43;
PNG
media_image2.png
355
322
media_image2.png
Greyscale
pg. 45, “
PNG
media_image3.png
153
311
media_image3.png
Greyscale
{Examiner correlates the offloading to determine the best course of action to be perform on the video processing in such that when the frame rate is below a certain threshold, the CNN can be performed on the mobile phone (lightweight DNN Model)}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, Lu, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, Lu, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 26, the modification of Yi, Liu, Ali, Lu, and Ran teaches claimed invention substantially as claimed, and Ran further teaches the determined configuration directs one or more cloud devices to perform processing of the extracted video frames using a heavy DNN model when results from the lightweight DNN model do not meet a threshold confidence value (Ran, pg. 44-45,
PNG
media_image4.png
397
492
media_image4.png
Greyscale
PNG
media_image3.png
153
311
media_image3.png
Greyscale
{Examiner correlates the offloading to determine the best course of action to be perform on the video processing in such that when the frame rate is above a certain threshold, the CNN can be performed on the server (heavyweight DNN Model)} while frame rates below the threshold of 4 Mbps is run locally on the phone}).
Regarding claim 33, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, Ali, and Lu does not explicitly teach the allocated processing of input data directs the set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices.
Ran teaches the allocated processing of input data directs the set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices (Ran,
pg. 43;
PNG
media_image5.png
246
488
media_image5.png
Greyscale
pg. 45, “
PNG
media_image3.png
153
311
media_image3.png
Greyscale
{Examiner correlates the offloading to determine the best course of action to be perform on the video processing in such that when the frame rate is below a certain threshold, the CNN can be performed on the mobile phone (lightweight DNN Model)}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, Lu, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, Lu, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 38, the modification of Yi, Liu, Ali, and Lu teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, Ali, and Lu does not explicitly teach the determined configuration directs the set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices.
Ran teaches the determined configuration directs the set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices (Ran, Introduction pg. 43;
PNG
media_image2.png
355
322
media_image2.png
Greyscale
pg. 45, “
PNG
media_image3.png
153
311
media_image3.png
Greyscale
{Examiner correlates the offloading to determine the best course of action to be perform on the video processing in such that when the frame rate is below a certain threshold, the CNN can be performed on the mobile phone (lightweight DNN Model)}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the teachings of Lu (teaches determined configuration directs the set of local devices to extract video frames from the live video stream using a decoding module) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, Lu, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, Lu, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 39, the modification of Yi, Liu, Ali, Lu, and Ran teaches claimed invention substantially as claimed, and Ran further teaches the determined configuration directs one or more cloud devices to perform processing of the extracted video frames using a heavy DNN model when results from the lightweight DNN model do not meet a threshold confidence value (Ran, pg. 45, “
PNG
media_image3.png
153
311
media_image3.png
Greyscale
{Examiner correlates the offloading to determine the best course of action to be perform on the video processing in such that when the frame rate is above a certain threshold, the CNN can be performed on the server (heavyweight DNN Model)}).
Claims 27, 29-30, and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature "LAVEA: Latency-aware Video Analytics on Edge Computing Platform" issued to Yi et al. (hereinafter as "Yi") in view of Non-Patent Literature “DARE: Dynamic Adaptive Mobile Augmented Reality with Edge Computing issued to Liu et al. (hereinafter as “Liu”) in view of Non-Patent Literature “Edge Enhanced Deep Learning Systems for Large-scale Video Stream Analytics” issued to Ali et al. (hereinafter as “Ali”) in further view of Non-Patent Literature "Delivering Deep Learning to Mobile Devices via Offloading" issued to Xukan Ran et al. (hereinafter as “Ran”).
Regarding claim 27, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, and Ali does not explicitly teach detecting a change in resource availability comprises determining whether network connectivity to the at least one edge device is available.
Ran teaches detecting a change in resource availability comprises determining whether network connectivity to the at least one edge device is available (Ran: 4.3 Impact of variable network conditions, pg. 45, “
PNG
media_image6.png
154
308
media_image6.png
Greyscale
{Examiner correlates determining the resource available based on the network connectivity according to determining the offload policy on whether the CNN on the phone runs better or the CNN on the server is suitable to run better}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 29, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, and Ali does not explicitly teach further allocating processing to one or more smart devices, the one or more smart devices performing processing that is computationally cheaper than processing performed by the at least one edge device.
Ran teaches further allocating processing to one or more smart devices, the one or more smart devices performing processing that is computationally cheaper than processing performed by the one or more edge devices (Ran: pg. 44-45, “
PNG
media_image7.png
228
329
media_image7.png
Greyscale
PNG
media_image8.png
355
325
media_image8.png
Greyscale
{Examiner correlates allocating the camera and edge devices based on the features received by adjusting the resolution of the camera according to the input and reviewing the results to determine that that lower resolution requires less computation while higher resolution require higher computation based on the associated learnable parameters of the device}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 30, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, and Ali does not explicitly teach dynamically shifting a processing load of the input data back to the one or more edge devices upon detecting that network capability between the set of local devices has been restored.
Ran teaches dynamically shifting the processing load of the input data back to the one or more edge devices upon detecting that network capability between the set of local devices has been restored (Ran: pg. 43
PNG
media_image5.png
246
488
media_image5.png
Greyscale
pg. 44-45, “
PNG
media_image6.png
154
308
media_image6.png
Greyscale
{Examiner correlates the offloading as determining to shift the processing load to increasing the processing of the mobile device by observing if the frame rate is below a threshold in which is then run on the mobile phone instead. The set of local devices is from offloading from a nearby device in such a smart phone based on the head mount display with camera would offload based on the frame rate requirement}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Ran (teaches the determined configuration directs set of local devices to perform processing of the extracted video frames using a lightweight DNN model locally on the set of local devices). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy
(See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Ran are directed to video analytic system based on offloading and awareness.
Regarding claim 40, the modification of Yi, Liu, and Ali teaches claimed invention substantially as claimed, however, the modification of Yi, Liu, and Ali does not explicitly teach detecting a change in resource availability comprises determining whether network connectivity to one or more cloud devices is available.
Ran teaches detecting a change in resource availability comprises determining whether network connectivity to one or more cloud devices is available (Ran: 4.3 Impact of variable network conditions, pg. 45, “
PNG
media_image6.png
154
308
media_image6.png
Greyscale
{Examiner correlates determining the resource available based on the network connectivity according to determining the offload policy on whether the CNN on the phone runs better or the CNN on the server is suitable to run better}).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify Yi (teaches receive a video query regarding a live video stream recorded by at least one camera; determine a resource requirement estimate for the video query; determine a set of configurations from a plurality of processing configurations, based upon an ability of each configuration of the set of configurations to process the video query; determine a configuration from the determined set of configurations for performing local processing of the video query based on an amount of available processing resources of at least one edge device, the at least one edge device positioned at a fixed location within an edge computing environment; allocate initial local processing for the video query between the at least one camera and the at least one edge device according to the determined configuration) with the teachings of Liu (teaches detect a change in resource availability for the determined configuration, wherein the change is at least one of: a device condition or a network condition) with the teachings of Ali (teaches when the detected change in resource availability is insufficient for the initial local processing of the video query, reallocate processing for the video query to one or more cloud, thereby continuing processing for the video query according to a different resource allocation) with the further teachings of Ran (teaches detecting a change in resource availability comprises determining whether network connectivity to one or more cloud devices is available). One of ordinary skill in the art would have been motivated to make such a combination of dramatically improving frames of the video by adjusting the resolution of the frame and improving the detection accuracy (See Ran: Section 3 METRICS AND DEGREES OF FREEDOM, pg. 44). In addition, the references (Yi, Liu, Ali, and Ran) teach features that are directed to analogous art and they are directed to the same field of endeavor as Yi, Liu, Ali, and Ran are directed to video analytic system based on offloading and awareness.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S Patent 11,503,113 issued Lijun Dong (hereinafter as “Dong”) teaches a method implemented by a network element (NE) in a mobile edge cloud (MEC) network in which offloading the request message from a client and utilizing a task related data to describe a task to offload the task to an edge cloud.
U.S Patent Application Publication 2018/0183855 issued to SABELLA et al. (hereinafter as “SABELLA”) teaches offloading computationally intensive task from one computer device to another based on energy consumption and latency budgets from the computations.
U.S Patent Application Publication 2013/0166711 issued to Wang et al. (hereinafter as “Wang”) teaches three-tier intelligent video surveillance management system that provides a gateway that allows to provided a filtered set of metadata to the cloud computing service.
U.S Patent 11,838,930 issued to Zhu et al. (hereinafter as “Zhu”) teaches multi-task joint computing service that unload and resource allocates the computing tasks from multiple computing.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW N HO whose telephone number is (571)270-0590. The examiner can normally be reached Tuesday and Thursday 10:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sherief Badawi can be reached at (571) 272-9782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
9/22/2026
/ANDREW N HO/Examiner
Art Unit 2169
/SHERIEF BADAWI/Supervisory Patent Examiner, Art Unit 2169
