DETAILED ACTION
This office action is in response to amendments filed on 01/05/2026.
Claims 1-5, 10-15, and 18-20 have been amended. Claims 8-9 and 16-17 have been canceled. Claims 1-7, 10-15, and 18-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Prior Art Rejections:
Applicant's arguments regarding the prior art rejections (pg. 13-15) have been fully considered but they are not persuasive.
Applicant argues that the amended independent claim limitations are not taught by any of the cited references. Applicant specifically argues that Prashant’s pod-based information processing system is not specific to the field of AI inference service for video stream processing, and thus fails to teach the limitation “the processing of the k-th channel of streaming media through the i-th inference service pod refers to: performing Artificial Intelligence (AI) inference on a current video frame of the k-th channel of streaming media obtained in real time and feeding back a real-time audit result…” Examiner respectfully notes that Prashant is not relied upon to teach this limitation. As can be seen in the rejection below, the limitation is taught by Canel, which explicitly discloses performing AI inference on video streams: “This paper presents FilterForward, a new edge-to-cloud system that enables datacenter-based applications to process content from thousands of cameras by installing lightweight edge filters that backhaul only relevant video frames. FilterForward introduces fast and expressive per-application ‘microclassifiers’ that share computation to simultaneously detect dozens of events on computationally constrained edge nodes” (Canel, Abstract).
The prior art rejections have been updated to include the amended limitations and to clarify
the reasoning given for the limitations that were not amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 10-15, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over
Canel et al. (hereinafter Canel), “Scaling Video Analytics on Constrained Edge Nodes” in view of
Prashant et al. (hereinafter Prashant), U.S. Patent Application Publication US 20220276929 A1.
Regarding Claim 1,
Canel teaches A streaming media processing method based on an inference service, comprising:
a process of processing a k-th channel of streaming media through an i-th inference service pod (Pg. 4, section 3.2: “Microclassifiers are lightweight binary classification neural networks that take as input feature maps extracted by the base DNN and output the probability that a frame is relevant to a particular application. An edge node can run many MCs [microclassifiers] on a single camera stream…” An edge node running a DNN feature extractor and microclassifiers is an i-th inference service pod, and a camera stream is a k-th channel of streaming media.)
wherein the processing of the k-th channel of streaming media through the i-th inference service pod refers to: performing Artificial Intelligence (Al) inference on a current video frame of the k-th channel of streaming media obtained in real time and feeding back a real-time audit result, based on the i-th inference service pod; (Pg. 1, section 1: “This paper addresses the question of how to overcome this network bottleneck and offload large volumes of data from a distributed camera deployment in real time to a datacenter for further processing.” Also see the portion of pg. 4, section 3.2 cited above. The DNN feature extractor and lightweight neural network microclassifiers (i.e. the i-th inference service pod) perform AI inference on a frame from the camera stream (i.e. a current video frame of the k-th channel of streaming media) in real time and output a probability of relevance (i.e. a real-time audit result).)
wherein the k-th channel of streaming media is any one of N channels of streaming media, the i-th inference service pod is any one of N inference service pods, and the N inference service pods are bound to the N channels of streaming media in one-to-one correspondence; wherein i and k are positive integers, and N is a positive integer greater than or equal to i and greater than or equal to k; (Pg. 1, Abstract: “This paper presents FilterForward, a new edge-to-cloud system that enables datacenter-based applications to process content from thousands of cameras by installing lightweight edge filters that backhaul only relevant video frames.” Pg. 4, section 3.2: “An edge node can run many MCs [microclassifiers] on a single camera stream…” Each edge node running a DNN feature extractor and microclassifiers is an inference service pod (e.g. the i-th inference service pod), and each camera stream is a channel of streaming media (e.g. the k-th channel of streaming media). The system can operate on thousands of cameras (i.e. N channels of streaming media), with each camera stream processed by its own edge node (i.e. there is a 1-to-1 correspondence between channels of streaming media and inference service pods).)
wherein the processing of the k-th channel of streaming media through the [replacement object of the] i-th inference service pod refers to: performing the Al inference on the current video frame of the k-th channel of streaming media obtained in real time and feeding back the real-time audit result, based on the [replacement object of the] i-th inference service pod. (See the portions of pg. 1, section 1 and pg. 4, section 3.2 cited above. The DNN feature extractor and lightweight neural network microclassifiers (i.e. the i-th inference service pod) perform AI inference on a frame from the camera stream (i.e. a current video frame of the k-th channel of streaming media) in real time and output a probability of relevance (i.e. a real-time audit result).)
Canel does not appear to explicitly disclose
detecting, [in a process of processing a k-th channel of streaming media through an i-th inference service pod], the i-th inference service pod, to obtain a detection result of the i-th inference service pod;
generating and sending replacement request information of the i-th inference service pod based on abnormal information of the i-th inference service pod, in a case of it is determined that the i-th inference service pod is in an abnormal state based on the detection result of the i-th inference service pod;
generating a replacement object of the i-th inference service pod, in response to obtaining replacement instruction information of the i-th inference service pod
processing the k-th channel of streaming media through the replacement object of the i-th inference service pod;
However, Prashant teaches detecting, [in a process of processing a k-th channel of streaming media through an i-th inference service pod], the i-th inference service pod, to obtain a detection result of the i-th inference service pod; ([0006]: “In one or more illustrative embodiments, a containerized workload is executed as a pod on a given node of the container environment.” [0004]: “The method determines when a first node executing at least one containerized workload in an information processing system has failed.” Detecting a node failure is obtaining a detection result of the containerized workload (i.e. pod) executed on the node.)
generating and sending replacement request information of the i-th inference service pod based on abnormal information of the i-th inference service pod, in a case of it is determined that the i-th inference service pod is in an abnormal state based on the detection result of the i-th inference service pod; ([0032]: “More particularly, illustrative embodiments force deletion of the one or more pods on a failed node, and cause creation and scheduling of replacement pods to healthy nodes.” [0046]: “By force-deleting the pod object (338-1), this allows the scheduler (114 in FIG. 1) to schedule creation of a replacement pod for the failed pod on another node.” A pod on a failed node is a pod in an abnormal state. Based on the detected node failure (i.e. abnormal information of the pod), the replacement pod is scheduled to be created on a healthy node (i.e. replacement request information is generated and sent).)
generating a replacement object of the i-th inference service pod, in response to obtaining replacement instruction information of the i-th inference service pod ([0046]: “By force-deleting the pod object (338-1), this allows the scheduler (114 in FIG. 1) to schedule creation of a replacement pod for the failed pod on another node. The stateful set controller (e.g., mentioned above as part of controller manager 112 of master node 110 in FIG. 1) creates pods on nodes, including replacement pods.” A replacement pod is created on another node (i.e. the replacement object of the pod is generated) in response to scheduling (i.e. replacement instruction information).)
processing the k-th channel of streaming media through the replacement object of the i-th inference service pod; ([0004]: “The method causes creation of a replacement containerized workload for execution on a second node in the information processing system.” Data is processed by the replacement containerized workload on the healthy second node (i.e. replacement object of the pod).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Canel and Prashant. Canel teaches processing data from a deployment of video cameras by equipping each data stream’s associated edge node with an inference model. Prashant teaches detecting and managing node failures in a distributed, pod-based information processing system. One of ordinary skill would have motivation to combine Canel and Prashant because Prashant’s node failure management techniques “enable, inter alia, resilience for application programs executing in the container environment” (Prashant, 0006), and “provide many benefits, by way of example only: (i) superior recovery times versus rebooting and removing the nodes; (ii) a robust mechanism for handling node cleanup; and prevention of new or replacement pods from being scheduled to a failed node until the node has regained health and has been cleaned up” (Prashant, 0053).
Regarding Claim 2, Canel and Prashant teach The method of claim 1, as shown above.
Canel also teaches further comprising:
Generating the N inference service pods, in a case of it is determined that there are the N channels of streaming media to be processed; and (Pg. 1, Abstract: “This paper presents FilterForward, a new edge-to-cloud system that enables datacenter-based applications to process content from thousands of cameras by installing lightweight edge filters that backhaul only relevant video frames.” Pg. 4, section 3.2: “An edge node can run many MCs [microclassifiers] on a single camera stream… To deploy an MC, the developer supplies the network weights and architecture specification along with the name of the base DNN layer (and, optionally, a crop thereof) to use as input.” An edge node running a DNN feature extractor and microclassifiers is an inference service pod, and a camera stream is a channel of streaming media. The system can operate on thousands of cameras (i.e. N channels of streaming media), with each camera stream processed by its own edge node (i.e. there is a 1-to-1 correspondence between channels of streaming media and inference service pods). Deploying MCs to an edge node amounts to generating an inference service pod of the N inference service pods.)
binding the N inference service pods to the N channels of streaming media, respectively, and (Pg. 1, section 1: “Using edge-compute resources collocated with the cameras, FilterForward identifies the video sequences that are most relevant to datacenter applications (‘filtering’) and offloads only that data for further analysis (‘forwarding’).” Figure 1 (pg. 3) shows a single instance of the FilterForward architecture, where a ‘Camera’ (i.e. a channel of streaming media) and a ‘Feature Extractor’ and MCs (i.e. an inference service pod) are collocated and bound together within an ‘Edge Node’. For a system with thousands of cameras (i.e. N channels of streaming media), this architecture will be duplicated thousands of times such that each camera stream is bound to one of thousands of inference services (i.e. N inference service pods).)
processing the N channels of streaming media through the N inference service pods, respectively. (Pg. 1, section 1: “Using edge-compute resources collocated with the cameras, FilterForward identifies the video sequences that are most relevant to datacenter applications (‘filtering’) and offloads only that data for further analysis (‘forwarding’).” Figure 1 (pg. 3) shows a single instance of the FilterForward architecture, where the arrows represent the flow of camera ‘Frames’ (i.e. streaming media) through the feature extractor and MCs (i.e. inference service pod), which perform filtering and forwarding (i.e. processing). For a system with thousands of cameras, this architecture will be duplicated thousands of times such that each camera stream (i.e. channel of the N channels of streaming media) is processed by its bound inference service (i.e. pod of the N inference service pods).)
Regarding Claim 3, Canel and Prashant teach The method of claim 2, as shown above.
Canel also teaches wherein generating the N inference service pods, comprises:
generating a j-th inference service pod of the N inference service pods, based on a j-th model inference service and a j-th group of configuration information, comprising: (Pg. 4, section 3.2: “An edge node can run many MCs [microclassifiers] on a single camera stream… To deploy an MC, the developer supplies the network weights and architecture specification…” Network weights and architecture specification are a j-th group of configuration information for the microclassifiers (i.e. model inference service) that will be deployed on an edge node to generate a j-th inference service pod.)
obtaining identification information of the j-th inference service pod of the N inference service pods, based on the j-th model inference service and the j-th group configuration information, wherein the identification information of the j-th inference service pod includes a name and/or serial number of the j-th inference service pod; and (Pg. 4, section 3.2: “To deploy an MC, the developer supplies… the name of the base DNN layer (and, optionally, a crop thereof) to use as input.” Generating the j-th inference service pod by deploying microclassifiers includes obtaining identification information including a name of the base DNN layer (i.e. of the j-th inference service pod).)
wherein j is a positive integer less than or equal to N, the j-th model inference service refers to an executable code of a j-th AI model inference service, and (Pg. 4, section 3.2: “Microclassifiers are lightweight binary classification neural networks that take as input feature maps extracted by the base DNN and output the probability that a frame is relevant to a particular application. An edge node can run many MCs on a single camera stream…” Microclassifiers (i.e. the model inference service) refer to neural networks (i.e. AI models) whose code is executed on an edge node.)
Prashant teaches acquiring connection information of the j-th inference service pod, based on the identification information of the j-th inference service pod, wherein the connection information of the j-th inference service pod includes at least one of: an Internet Protocol, IP, address of the j-th inference service pod, or a uniform resource locator, URL, address of the j-th inference service pod; (0015: “Pods may each have a unique Internet Protocol (IP) address enabling pods to communicate with one another, and for other system components to communicate with each pod.”)
the j-th group of configuration information comprises at least one of: probing interface configuration information of the j-th inference service pod used to configure a probing interface for the j-th inference service pod, resource restriction configuration information of the j-th inference service pod including a range of system resources occupied by the j-th inference service pod, scheduling configuration information of the j-th inference service pod including address information of a physical machine running the j-th inference service pod, or storage configuration information of the j-th inference service pod including a data storage location of the j-th inference service pod. ([0015]: “A pod may be configured as a single process wherein one or more containers execute one or more functions that operate together to implement the process… Configuration information (configuration objects) indicating how a container executes can be specified for each pod.” [0037]: “API objects 303 depict various configuration objects that are managed during system operations. For example, as shown and as will be further explained below, API objects 303 comprise volume attachment objects 332-1 and 332-2. Volume attachment object 332-1 defines the correspondence between V1 at node 310-1 and V1 in storage array 320. Similarly, volume attachment object 332-2 defines the correspondence between V2 at node 310-1 and V2 in storage array 320.” A pod is configured using configuration objects (i.e. configuration information), including volume attachment objects defining correspondence between the node and a location in storage (i.e. storage configuration information including a data storage location of the inference service pod).)
Regarding Claim 4, Canel and Prashant teach The method of claim 1, as shown above.
Prashant also teaches wherein detecting the i-th inference service pod to obtain the detection result of the i-th inference service pod, comprises:
performing an interface probing detection on the i-th inference service pod to obtain the detection result of the i-th inference service pod; and ([0030]: “…the Kubernetes system will detect a node failure in approximately 30 seconds using a heartbeat detection mechanism…” A heartbeat detection mechanism is a probing detection mechanism.)
determining that the i-th inference service pod is in the abnormal state, based on the detection result of the i-th inference service pod, comprises: determining that the i-th inference service pod is in the abnormal state, in the case of the detection result of the i-th inference service pod is a first type result, wherein the first type result comprises a connection failure or a connection error. ([0030]: “Although the Kubernetes system will detect a node failure in approximately 30 seconds using a heartbeat detection mechanism, Kubernetes cannot be certain whether the node has failed because it has ceased to operate or whether the node has experienced a loss of communication with the rest of the Kubernetes cluster.” A node failure is a result type, and this result type includes the case of a loss of communication (i.e. connection failure).)
Claim 5 recites identical limitations to those of claim 4, but depends on claim 2 rather than claim 1. Canel and Prashant teach The method of claim 2, as well as the additional limitations of claim 4, as shown above.
Regarding Claim 6, Canel and Prashant teach The method of claim 1, as shown above.
Prashant also teaches wherein processing the k-th channel of streaming media through the replacement object of the i-th inference service pod, comprises:
generating and sending replacement request information of the i-th inference service pod, based on abnormal information of the i-th inference service pod and related information of the replacement object of the i-th inference service pod; and ([0032]: “More particularly, illustrative embodiments force deletion of the one or more pods on a failed node, and cause creation and scheduling of replacement pods to healthy nodes.” [0017]: “Scheduler 114 typically schedules pods to run on particular nodes taking into account node resources and application execution requirements…” [0046]: “By force-deleting the pod object (338-1), this allows the scheduler (114 in FIG. 1) to schedule creation of a replacement pod for the failed pod on another node.” Based on the detected node failure (i.e. abnormal information of the pod) and taking into account node resources (i.e. related information of a replacement object), the replacement pod is scheduled to be created on a healthy node (i.e. replacement request information is generated and sent).)
processing the k-th channel of streaming media through the replacement object of the i-th inference service pod, in response to obtaining replacement instruction information of the i-th inference service pod. ([0004]: “The method causes creation of a replacement containerized workload for execution on a second node in the information processing system.” Data is processed by the replacement containerized workload on the healthy second node (i.e. replacement object of the pod) in response to scheduling (i.e. replacement instruction information).)
Claim 7 recites identical limitations to those of claim 6, but depends on claim 2 rather than claim 1. Canel and Prashant teach The method of claim 2, as well as the additional limitations of claim 6, as shown above.
Regarding Claim 10, Canel and Prashant teach The method of claim 2, as shown above.
Prashant also teaches wherein after generating the replacement object of the i-th inference service pod, the method further comprises:
disconnecting a binding between the k-th channel of streaming media and the i-th inference service pod, ([0019]: “The storage volumes depicted in the storage devices of storage array 205 can include any data generated in the information processing system 200 but, more typically, include data generated, manipulated, or otherwise accessed, during the execution of one or more applications in the nodes of host devices 202.” [0035]: “Also assume that two mapped volumes 316 (V1 and V2) are associated with pod 312-1.” [0044]: “As shown in step 35 of FIG. 3E, controller-podmon 342 then contacts storage array 320, in some embodiments through a CSI driver, to fence the volumes (V1 and V2) used by pod 312-1 on the failed node 310-1.” Volumes V1 and V2 hold the data that is accessed by the pod (i.e. the k-th channel of streaming media), and when the pod’s node fails, the volumes are fenced (i.e. disconnected).)
binding the k-th channel of streaming media with the replacement object of the i-th inference service pod, and ([0046]: “Thus, as shown in step 37 of FIG. 3G, in accordance with information in stateful set object 336, the stateful set controller creates a replacement of pod 312-1 on node 310-2 in the form of pod 312-2, including mapping volumes V1 and V2 to node 312-2.” The volumes holding the data that is accessed by the pod (i.e. the k-th channel of streaming media) are mapped (i.e. bound) to the replacement pod/node (i.e. the replacement object).)
deleting information storage record of the i-th inference service pod. ([0037]: “Volume attachment object 332-1 defines the correspondence between V1 at node 310-1 and V1 in storage array 320. Similarly, volume attachment object 332-2 defines the correspondence between V2 at node 310-1 and V2 in storage array 320.” [0045]: “Turning now to step 36 of FIG. 3F, after fencing the volumes, the controller-podmon 342 then deletes the volume attachment objects 332-1 and 332-2, and the pod object 338-1, associated with pod 312-1 of failed node 310-1.” The volume storage attachments (i.e. information storage record) of the failed pod/node are deleted.)
Claim 11 recites identical limitations to those of claim 10, but depends on claim 3 rather than claim 2. Canel and Prashant teach The method of claim 3, as well as the additional limitations of claim 10, as shown above.
Claim 12 recites identical limitations to those of claim 10, but depends on claim 4 rather than claim 2. Canel and Prashant teach The method of claim 4, as well as the additional limitations of claim 10, as shown above.
Claim 13 recites identical limitations to those of claim 10, but depends on claim 5 rather than claim 2. Canel and Prashant teach The method of claim 5, as well as the additional limitations of claim 10, as shown above.
Claim 14 recites identical limitations to those of claim 10, but depends on claim 6 rather than claim 2. Canel and Prashant teach The method of claim 6, as well as the additional limitations of claim 10, as shown above.
Claim 15 recites identical limitations to those of claim 10, but depends on claim 7 rather than claim 2. Canel and Prashant teach The method of claim 7, as well as the additional limitations of claim 10, as shown above.
Claims 18 and 19 are system claims containing substantially the same elements as method claims 1 and 2, respectively. Canel and Prashant teach the elements of claims 1 and 2, as shown above.
Prashant also teaches An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores an instruction executable by the at least one processor, and the instruction, when executed by the at least one processor, enables the at least one processor to execute: ([0070]: “As indicated previously, components of an information processing system as disclosed herein can be implemented at least in part in the form of one or more software programs stored in memory and executed by a processor of a processing device.”)
Claim 20 is a product claim containing substantially the same elements as method claim 1. Canel and Prashant teach the elements of claim 1, as shown above.
Prashant also teaches A non-transitory computer-readable storage medium storing a computer instruction thereon, wherein the computer instruction is used to cause a computer to execute: ([0005]: “Further illustrative embodiments are provided in the form of a non-transitory computer-readable storage medium having embodied therein executable program code that when executed by a processor causes the processor to perform the above steps.”)
Conclusion
Claims 1-7, 10-15, and 18-20 are rejected.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN M ROHD whose telephone number is (571)272-6445. The examiner can normally be reached Mon-Thurs 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.M.R./Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights