Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on July 13, 2026, in which claims 1, 7-9, 15, 17, and 18 are amended. No claims have been newly cancelled or added. Claims 1-19 and 21 are currently pending.
Response to Arguments
With regards to the rejections of claims 1-20 under 35 U.S.C. 101 as being directed towards abstract ideas, Applicant argues that the claims as amended are patentable at least at Step 2A, Prong Two, by integrating any recited judicial exceptions into a practical application. Examiner finds Applicant’s arguments persuasive, the details presented in the independent claims as amended recite a special purpose computer with a practical usage.
With regards to the rejection of claims 1-7, 9-15, and 17-19 under 35 U.S.C. 103 as unpatentable over Liu et al. “DARTS: Differentiable Architecture Search”, in view of Rabinovich et al. (U.S. Patent Application Publication No. 2021/0182636), Applicant argues that the claims as amended recite limitations that are not taught by the combination of Liu and Rabinovich. Applicant argues with respect to claim 1. Examiner respectfully disagrees.
Applicant argues specifically on page 19 of the Remarks that the newly amended limitation a quantity of the plurality of structuring elements is determined based on graphics processing unit memory resources of the first device. While acknowledging that Liu “refers to a number of cells in the stack that can be increased from 8 to 20” on Liu Pg. 13, Applicant asserts “Liu has not been shown to teach or suggest that the number of cells in the stack is determined based on graphics processing unit memory resources of a device”.
While Examiner agress that Liu does not teach, for instance, what is taught in paragraph [0180] of the specification:
“when there are only a few graphics processing unit memory resources of the neural architecture search apparatus performing the method shown in FIG. 7, there can be a smaller quantity of structuring elements, but when there are abundant graphics processing unit memory resources of the neural architecture search apparatus performing the method shown in FIG. 7, there can be a larger quantity of structuring elements”.
In contrast, the newly amended limitation a quantity of the plurality of structuring elements is determined based on graphics processing unit memory resources of the first device has broader scope. The limitation does not recite that the quantity should be changed based on the available memory resources of the graphics processing unit (GPU), merely that there is a quantity (i.e. a number) of the structuring elements, and that this quantity is determined based on GPU memory resources. Liu teaches:
(Liu Pg. 12) “A small network of 8 cells is trained using DARTS for 50 epochs, with batch size 64 (for both the training and validation sets) and the initial number of channels 16. The numbers were chosen to ensure the network can fit into a single GPU”.
So, Liu teaches a quantity of the plurality of structuring elements is determined (the number of the cells being 8) and that this quantity is determined based on graphics processing unit memory resources of the first device (as the number of cells, along with the number of epochs, the batch size, and the initial number of channels, were chosen so that a single GPU could run their network). GPUs inherently contain memory resources such as cache and, oftentimes, RAM, and Liu teaches this more explicitly at:
(Liu Pg. 13) “we conducted architecture search on CIFAR-10 by increasing the number of cells in the stack from 8 to 20. The initial number of channels is reduced from 16 to 6 due to memory budget of a single GPU”.
Above Liu also discloses that the constraint on the number of cells and channels is due to the memory resources available on the GPU, as when the number of cells increases, the number of channels must decrease to fit the memory budget. Therefore, Liu teaches the relevant amended limitation of claim 1, and as shown in the rejection below, the combination of Liu and Rabinovich teaches claim 1 entirely. Liu and Rabinovich also teach the other independent claims, claims 7, 9, 15, 17, and 18, under similar rationales, as shown in the rejections below.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections - Allowable Subject Matter
Claims 8 and 16 have no outstanding rejection over the cited prior art. The closest identified art is Liu et al. “DARTS: Differentiable Architecture Search”. Within, at Liu Pg. 4, Algorithm 1, formulas for optimizing an architecture are recited, in the “while not converged” loop:
PNG
media_image1.png
217
900
media_image1.png
Greyscale
However, these formulas recited by Liu do not teach both formulas recited in both claims 8 and 16:
PNG
media_image2.png
85
262
media_image2.png
Greyscale
Both the formulas of Liu and the formulas of claims 8 and 16 recite calculation of an architecture parameter α and a weight parameter w. Additionally, they share some terms verbatim such as Ltrain, and some terms which appear to have equivalent functionality, such as ξ in Liu and ∂α in claim 8/16. However, at least the formula for calculating a weight parameter of claim 8/16 is substantially different from the corresponding formula in Liu, Liu’s formula lacking at least an equivalent to the term ∂w. Additionally, Liu’s formula for the architecture parameter α uses terms relative to the weights in several places where the corresponding formula of claim 8/16 uses terms relative to the architecture parameter itself. For at least the above reasons, the reference does not teach the subject matter of claims 8 and 16.
A search for the prior arts with PE2E Search and ip.com InnovationQ+ has been conducted. Besides patent databases, searching over non-patent literature databases, such as Google Scholar and ApproachZero has also been performed. The prior arts searched and investigated in patent and non-patent domains do not fairly teach or suggest the above limitations recited in claims 8 and 16.
Therefore, claims 8 and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Prior Art
The following references are used for prior art claim rejections:
Liu et al. “DARTS: Differentiable Architecture Search”, hereinafter Liu
Rabinovich et al. (U.S. Patent Application Publication No. 2021/0182636), hereinafter Rabinovich
Spangenberg (U.S. Patent Application Publication No. 2018/0257641), hereinafter Spangenberg
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Rabinovich.
Regarding claim 1,
Liu teaches A method, comprising:
constructing, by a first device, a target neural network for processing image or voice data ((Liu Pg. 13) “We consider the mobile setting where the input image size is 224×224…A network of 14 cells is trained for 250 epochs…The training takes 12 days on a single GPU”) based on optimized structuring elements, (Liu Pg. 4, Algorithm 1 shows that the last step is deriving the final neural network architecture based on previously optimized weights for the structuring elements) wherein:
PNG
media_image1.png
217
900
media_image1.png
Greyscale
a search space and a plurality of structuring elements ((Liu Pg. 2) “DARTS is able to learn high-performance architecture building blocks with complex graph topologies within a rich search space”, architecture building blocks correspond to structuring elements) are used for processing the image or the voice data in the target neural network, ((Liu Abstract) “This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner…our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification”)
a quantity of the plurality of structuring elements is determined based on graphics processing unit memory resources of the first device, ((Liu Pg. 12) “To carry out architecture search, we hold out half of the CIFAR-10 training data as the validation set. A small network consisting of 8 cells is trained using DARTS for 50 epochs, with batch size 64 (for both the training and validation sets) and the initial number of channels 16. The numbers were chosen to ensure the network can fit into a single GPU”, (Liu Pg. 13) “To better understand the effect of depth for architecture search, we conducted architecture search on CIFAR-10 by increasing the number of cells in the stack from 8 to 20. The initial number of channels is reduced from 16 to 6 due to memory budget of a single GPU”, architecture search with quantities of 8 cells or 20 cells using a single GPU with a memory budget, wherein the specific numbers are chosen so that the network can fit into the GPU, corresponds to determination of quantities of pluralities of structuring elements based on graphics processing unit memory resources of a first device)
the search space comprises a plurality of groups of alternative operators, ((Liu Pg. 3) “Let O be a set of candidate operations (e.g., convolution, max pooling, zero) where each operation represents some function o(·) to be applied to x(i). To make the search space continuous, we relax the categorical choice of a particular operation to a softmax over all possible operations”)
operators in each group of alternative operators are of a same type, ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, group types include separable convolutions, dilated separable convolutions, pooling, identity, and zero)
each of the plurality of structuring elements is a network structure between a plurality of nodes obtained by connecting basic operators of a neural network, ((Liu Pg. 2) “Following Zoph et al. (2018); Real et al. (2018); Liu et al. (2018a;b), we search for a computation cell as the building block of the final architecture. The learned cell could either be stacked to form a convolutional network or recursively connected to form a recurrent network. A cell is a directed acyclic graph consisting of an ordered sequence of N nodes. Each node x(i) is a latent representation (e.g. a feature map in convolutional networks) and each directed edge (i, j) is associated with some operation o(i,j) that transforms x(i)”, computation cells correspond to structuring elements, each computation cell is a network structure (acyclic graph) between a plurality of nodes, each edge connecting the nodes is associated with an operation, which corresponds to connecting basic operators)
and the nodes of each of the plurality of structuring elements are connected to form an edge; (Liu Pg. 6, Fig. 4 shows an example of a cell, i.e. a structuring element, with nodes connected to form edges)
PNG
media_image3.png
225
401
media_image3.png
Greyscale
an initial neural architecture at a first stage in the target neural network is based on the plurality of structuring elements being stacked, ((Liu Pg. 5) “Our convolutional cell consists of N = 7 nodes, among which the output node is defined as the depthwise concatenation of all the intermediate nodes (input nodes excluded). The rest of the setup follows Zoph et al. (2018); Liu et al. (2018a); Real et al. (2018), where a network is then formed by stacking multiple cells together”, a network corresponds to an initial neural architecture)
each edge of each structuring element in the initial neural architecture at the first stage corresponds to a plurality of alternative operators, (Liu Pg. 3, Fig. 1 shows that the initial neural architecture has edges that are initially unknown have a mixture of candidate operations placed on them, which corresponds to a plurality of alternative operators)
PNG
media_image4.png
457
886
media_image4.png
Greyscale
and each of the plurality of alternative operators corresponds to one group in the plurality of groups of alternative operators; ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, group types include separable convolutions, dilated separable convolutions, pooling, identity, and zero)
an optimized initial neural architecture at the first stage in the target neural network is based on the initial neural architecture at the first stage being optimized to be convergent; (Liu Pg. 4, Algorithm 1 shows an optimization procedure that continues until the architecture is convergent, with each iteration of the optimization procedure increasing the convergence until the model is considered convergent)
a mixed operator corresponding to a jth edge of an ith structuring element in an initial neural architecture at a second stage in the target neural network ((Liu Pg. 3) “the operation mixing weights for a pair of nodes (i, j)… At the end of search, a discrete architecture can be obtained by replacing each mixed operation ō(i,j) with the most likely operation”) comprises all operators in a kth group of alternative operators in the optimized initial neural architecture at the first stage, ((Liu Pgs. 4-5) “To form each node in the discrete architecture, we retain the top-k strongest operations (from distinct nodes) among all non-zero candidate operations collected from all the previous nodes…To make our derived architecture comparable with those in the existing works, we use k = 2 for convolutional cells”, each node retaining both of the top 2 strongest operations, with the mixed operation being replaced with the most likely operation, corresponds to a mixed operator corresponding to a structuring element comprising all operators in a kth group)
the kth group of alternative operators is a group of alternative operators comprising an operator with a largest weight in a plurality of alternative operators corresponding to the jth edge of the ith structuring element in the optimized initial neural architecture at the first stage, ((Liu Pg. 4) “To form each node in the discrete architecture, we retain the top-k strongest operations (from distinct nodes) among all non-zero candidate operations collected from all the previous nodes. The strength of an operation is defined as [following equation]”, a strongest operation corresponds to an operator with a largest weight as defined by the strength/weighting equation, the equation shows that the operators correspond to a plurality of elements (i,j) within the architecture)
PNG
media_image5.png
177
558
media_image5.png
Greyscale
and i, j, and k are all positive integers; ((Liu Pgs. 4-5) “To make our derived architecture comparable with those in the existing works, we use k = 2 for convolutional cells”, (Liu Pg. 3) “the operation mixing weights for a pair of nodes (i, j)”)
and the optimized structuring elements in the target neural network are based on the initial neural architecture at the second stage being optimized to be convergent; (Liu Pg. 4, Algorithm 1 shows an optimization procedure that continues until the architecture is convergent, then deriving a final architecture with optimized weights for the structuring elements)
Rabinovich teaches the following further limitations more explicitly than Liu or that Liu does not teach:
receiving, by the first device and from a second device, to-be-processed image data or to-be processed voice data; ((Rabinovich [0032]) “In some embodiments, the system is employed to implement computer vision functionality. As such, the system may include one or more image capture devices, such as camera 103, to capture image data 101 for one or more objects 105 in the environment at which the system operates. The image data 101 and/or any analysis results (e.g., classification output data 113) may be stored in one or more computer readable storage mediums”)
processing, by the first device, the to-be-processed image data or the to-be-processed voice data using the target neural network to obtain a result of processing the to-be-processed image data or the to-be-processed voice data; ((Rabinovich [0031]) “FIG. 1 illustrates an example system which may be employed in some embodiments of the invention to implement structure learning for neural networks”, (Rabinovich [0032]) “In some embodiments, the system is employed to implement computer vision functionality. As such, the system may include one or more image capture devices, such as camera 103, to capture image data 101 for one or more objects 105 in the environment at which the system operates. The image data 101 and/or any analysis results (e.g., classification output data 113) may be stored in one or more computer readable storage mediums”)
PNG
media_image6.png
815
1142
media_image6.png
Greyscale
and transmitting, by the first device and to the second device, the result of processing the to-be processed image data or the to-be-processed voice data ((Rabinovich [0032]) “The computer readable storage medium includes any combination of hardware and/or software that allows for ready access to the data that is located at the computer readable storage medium. For example, the computer readable storage medium could be implemented as…remote storage in a networked storage device, such as networked attached storage (NAS), storage area network (SAN), or cloud storage”, use of remote storage in a networked device includes transmission of data to the remote device)
At the time of filing, one of ordinary skill in the art would have motivation to combine Liu and Rabinovich by taking the method for neural architecture search taught by Liu and combining it with the structure learning method including transmission of image data between a neural network structure learning system and a remote storage medium, with processing of the data at the system, taught by Rabinovich, as use of remote storage, such as cloud storage, imparts increased flexibility in adjusting the amount of storage available to be precisely as much as is needed for the current application of the method. Such a combination would be obvious.
Regarding claim 2,
Liu and Rabinovich jointly teach The method according to claim 1, wherein the method further comprises:
Liu further teaches:
performing clustering on a plurality of alternative operators in the search space, to obtain the plurality of groups of alternative operators ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, separable convolutions and dilated separable convolutions are grouped together, grouping corresponds to performing clustering)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Liu and Rabinovich for the parent claim of claim 2, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 3,
Liu and Rabinovich jointly teach The method according to claim 1, wherein the method further comprises:
Liu further teaches:
selecting one operator from each of the plurality of groups of alternative operators, to obtain the plurality of alternative operators corresponding to each edge of each structuring element in the initial neural architecture at the first stage ((Liu Pg. 3) “The task of learning the cell therefore reduces to learning the operations on its edges…Let O be a set of candidate operations (e.g., convolution, max pooling, zero) where each operation represents some function o(·) to be applied to x(i). To make the search space continuous, we relax the categorical choice of a particular operation to a softmax over all possible operations”, the candidate operations for each structuring element x(i) include all possible operations, therefore at least one operator from each group would be selected)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Liu and Rabinovich for the parent claim of claim 3, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 4,
Liu and Rabinovich jointly teach The method according to claim 3, wherein the method further comprises:
Liu further teaches:
determining an operator with a largest weight on each edge of each structuring element in the initial neural architecture at the first stage; ((Liu Pg. 3) “the operation mixing weights for a pair of nodes (i, j) are parameterized by a vector α(i,j) of dimension |O|…At the end of search, a discrete architecture can be obtained by replacing each mixed operation o(i,j) with the most likely operation”, a most likely operation for the edge between two nodes at the end of search corresponds to determining an operator with a largest weight for the edge between each structuring element at the first stage)
and determining a mixed operator comprising all alternative operators in a group of alternative operators ((Liu Pg. 3) “To make the search space continuous, we relax the categorical choice of a particular operation to a softmax over all possible operations: [Equation 2]…At the end of search, a discrete architecture can be obtained by replacing each mixed operation o(i,j)”, a mixed operation that encompasses all possible operations corresponds to a mixed operator comprising all alternative operators within a group) in which there is an operator with a largest weight on a jth edge of an ith structuring element in the initial neural architecture at the first stage as an alternative operator corresponding to the jth edge of the ith structuring element in the initial neural architecture at the second stage (Liu Pg. 3, Fig. 1 shows that the operators on the edges of the initial neural architecture with the largest weights at step (c) are selected to be the operators at step (d), step (c) corresponds to part of the first stage and step (d) corresponds to part of the second stage)
PNG
media_image7.png
149
1133
media_image7.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Liu and Rabinovich for the parent claim of claim 4, claim 3. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 5,
Liu and Rabinovich jointly teach The method according to claim 1, wherein the plurality of groups of alternative operators comprise:
Liu further teaches:
a first group of alternative operators, comprising 3 x 3 max pooling and 3 x 3 average pooling; ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, 3 × 3 max pooling and 3 × 3 average pooling are operations considered to be operators in cells)
a second group of alternative operators, comprising a skip connection; ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, identity operations are considered to be operators in cells, an identity operator is alternative terminology for a skip connection, as disclosed by the specification, [0164] “a second group of alternative operators, including a skip connection (identity or skip-connect)”)
a third group of alternative operators, comprising 3 x 3 separable convolutions and 5 x 5 separable convolutions; ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, 3 × 3 and 5 × 5 separable convolutions are operations considered to be operators in cells)
and a fourth group of alternative operators, comprising 3 x 3 dilated separable convolutions and 5 x 5 dilated separable convolutions ((Liu Pg. 5) “We include the following operations in O: 3 × 3 and 5 × 5 separable convolutions, 3 × 3 and 5 × 5 dilated separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, identity, and zero”, 3 × 3 and 5 × 5 dilated separable convolutions are operations considered to be operators in cells)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Liu and Rabinovich for the parent claim of claim 5, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 6,
Liu and Rabinovich jointly teach The method according to claim 1, wherein the optimizing the initial neural architecture at the first stage to be convergent, to obtain optimized structuring elements comprises at least one of:
Liu further teaches:
separately optimizing, by using same training data, a network architecture parameter and a network model parameter that are of a structuring element in the initial neural architecture at the first stage to be convergent, to obtain the optimized initial neural architecture at the first stage;
or separately optimizing, by using same training data, a network architecture parameter and a network model parameter (Liu Pg. 4, Algorithm 1 shows a network architecture parameter α and a network model parameter w representing weights being updated in two separate steps) that are of a structuring element in the initial neural architecture ((Liu Pg. 3) “the operation mixing weights for a pair of nodes (i, j) are parameterized by a vector α(i,j) of dimension |O|. The task of architecture search then reduces to learning a set of continuous variables α = {α(i,j)}”, the weights and architecture α parameterize the nodes, which are structuring elements) at the second stage to be convergent, (Liu Pg. 4, Algorithm 1 shows that the optimization algorithm runs until convergence is achieved) to obtain the optimized structuring elements (Liu Pg. 3, Fig. 1 shows that at step (c) optimization of the structuring elements takes place, leading to the final architecture with optimized structuring elements at step (d))
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Liu and Rabinovich for the parent claim of claim 6, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 7,
Liu teaches A method, comprising:
constructing, by a first device, a target neural network for processing image or voice data ((Liu Pg. 13) “We consider the mobile setting where the input image size is 224×224…A network of 14 cells is trained for 250 epochs…The training takes 12 days on a single GPU”) based on optimized structuring elements, (Liu Pg. 4, Algorithm 1 shows that the last step is deriving the final neural network architecture based on previously optimized weights for the structuring elements) wherein:
a search space and a plurality of structuring elements ((Liu Pg. 2) “DARTS is able to learn high-performance architecture building blocks with complex graph topologies within a rich search space”, architecture building blocks correspond to structuring elements) are used for processing the image or the voice data in the target neural network, ((Liu Abstract) “This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner…our method is based on the continuous relaxation of the architecture representation, allowing efficient search of the architecture using gradient descent. Extensive experiments on CIFAR-10, ImageNet, Penn Treebank and WikiText-2 show that our algorithm excels in discovering high-performance convolutional architectures for image classification”)
a quantity of the plurality of structuring elements is determined based on graphics processing unit memory resources of the first device, ((Liu Pg. 12) “To carry out architecture search, we hold out half of the CIFAR-10 training data as the validation set. A small network consisting of 8 cells is trained using DARTS for 50 epochs, with batch size 64 (for both the training and validation sets) and the initial number of channels 16. The numbers were chosen to ensure the network can fit into a single GPU”, (Liu Pg. 13) “To better understand the effect of depth for architecture search, we conducted architecture search on CIFAR-10 by increasing the number of cells in the stack from 8 to 20. The initial number of channels is reduced from 16 to 6 due to memory budget of a single GPU”, architecture search with quantities of 8 cells or 20 cells using a single GPU with a memory budget, wherein the specific numbers are chosen so that the network can fit into the GPU, corresponds to determination of quantities of pluralities of structuring elements based on graphics processing unit memory resources of a first device)
and each of the plurality of structuring elements is a network structure between a plurality of nodes obtained by connecting basic operators of a neural network; ((Liu Pg. 2) “Following Zoph et al. (2018); Real et al. (2018); Liu et al. (2018a;b), we search for a computation cell as the building block of the final architecture. The learned cell could either be stacked to form a convolutional network or recursively connected to form a recurrent network. A cell is a directed acyclic graph consisting of an ordered sequence of N nodes. Each node x(i) is a latent representation (e.g. a feature map in convolutional networks) and each directed edge (i, j) is associated with some operation o(i,j) that transforms x(i)”, computation cells correspond to structuring elements, each computation cell is a network structure (acyclic graph) between a plurality of nodes, each edge connecting the nodes is associated with an operation, which corresponds to connecting basic operators)
a search network in the target neural network is based on the plurality of structuring elements being stacked; ((Liu Pg. 5) “Our convolutional cell consists of N = 7 nodes, among which the output node is defined as the depthwise concatenation of all the intermediate nodes (input nodes excluded). The rest of the setup follows Zoph et al. (2018); Liu et al. (2018a); Real et al. (2018), where a network is then formed by stacking multiple cells together”, a network corresponds to a search network)
the optimized structuring elements in the target neural network ((Liu Pg. 5) “In the first stage, we search for the cell architectures using DARTS, and determine the best cells based on their validation performance”, (Liu Pg. 2) “A cell is a directed acyclic graph consisting of an ordered sequence of N nodes”, determining the best cells, which consist of nodes, correspond to obtaining optimized structuring elements) are based on a network architecture parameter and a network model parameter, (Liu Pg. 4, Algorithm 1 shows a network architecture parameter α and a network model parameter w) that are of the structuring elements in the search network, ((Liu Pg. 3) “the operation mixing weights for a pair of nodes (i, j) are parameterized by a vector α(i,j) of dimension |O|. The task of architecture search then reduces to learning a set of continuous variables α = {α(i,j)}”, the weights and architecture α parameterize the nodes, which are structuring elements) being separately optimized in the search space by using same training data; (Liu Pg. 4, Algorithm 1 shows that network architecture parameter α and network model parameter w representing weights are updated in two separate steps)
Rabinovich teaches the following further limitations more explicitly than Liu or that Liu does not teach:
receiving, by the first device and from a second device, to-be-processed image data or to-be processed voice data; ((Rabinovich [0032]) “In some embodiments, the system is employed to implement computer vision functionality. As such, the system may include one or more image capture devices, such as camera 103, to capture image data 101 for one or more objects 105 in the environment at which the system operates. The image data 101 and/or any analysis results (e.g., classification output data 113) may be stored in one or more computer readable storage mediums”)
processing, by the first device, the to-be-processed image data or the to-be-processed voice data using the target neural network to obtain a result of processing the to-be-processed image data or the to-be-processed voice data; ((Rabinovich [0031]) “FIG. 1 illustrates an example system which may be employed in some embodiments of the invention to implement structure learning for neural networks”, (Rabinovich [0032]) “In some embodiments, the system is employed to implement computer vision functionality. As such, the system may include one or more image capture devices, such as camera 103, to capture image data 101 for one or more objects 105 in the environment at which the system operates. The image data 101 and/or any analysis results (e.g., classification output data 113) may be stored in one or more computer readable storage mediums”)
and transmitting, by the first device and to the second device, the result of processing the to-be processed image data or the to-be-processed voice data ((Rabinovich [0032]) “The computer readable storage medium includes any combination of hardware and/or software that allows for ready access to the data that is located at the computer readable storage medium. For example, the computer readable storage medium could be implemented as…remote storage in a networked storage device, such as networked attached storage (NAS), storage area network (SAN), or cloud storage”, use of remote storage in a networked device includes transmission of data to the remote device)
At the time of filing, one of ordinary skill in the art would have motivation to combine Liu and Rabinovich by taking the method for neural architecture search taught by Liu and combining it with the structure learning method including transmission of image data between a neural network structure learning system and a remote storage medium, with processing of the data at the system, taught by Rabinovich, as use of remote storage, such as cloud storage, imparts increased flexibility in adjusting the amount of storage available to be precisely as much as is needed for the current application of the method. Such a combination would be obvious.
Regarding claim 9,
Claim 9 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 1. Specifically, claim 9 recites An apparatus, comprising: at least one processor; and a memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to: [perform the method of claim 1].
Rabinovich recites: (Rabinovich [0079]) “FIG. 7 is a block diagram of an illustrative computing system 1400 suitable for implementing an embodiment of the present invention. Computer system 1400 includes a bus 1406 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 1407, system memory 1408 (e.g., RAM)”.
All other limitations in claim 9 are substantially the same as those in claim 1, therefore the same rationale for rejection applies.
Regarding claim 10,
Claim 10 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 2. All other limitations in claim 10 are substantially the same as those in claim 2, therefore the same rationale for rejection applies.
Regarding claim 11,
Claim 11 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 3. All other limitations in claim 11 are substantially the same as those in claim 3, therefore the same rationale for rejection applies.
Regarding claim 12,
Claim 12 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 4. All other limitations in claim 12 are substantially the same as those in claim 4, therefore the same rationale for rejection applies.
Regarding claim 13,
Claim 13 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 5. All other limitations in claim 13 are substantially the same as those in claim 5, therefore the same rationale for rejection applies.
Regarding claim 14,
Claim 14 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 6. All other limitations in claim 14 are substantially the same as those in claim 6, therefore the same rationale for rejection applies.
Regarding claim 15,
Claim 15 recites an apparatus comprising a processor and a memory storing computer-readable instructions for performing the function of the method of claim 7. Specifically, claim 15 recites An apparatus, comprising: at least one processor; and a memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to: [perform the method of claim 7].
Rabinovich recites: (Rabinovich [0079]) “FIG. 7 is a block diagram of an illustrative computing system 1400 suitable for implementing an embodiment of the present invention. Computer system 1400 includes a bus 1406 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 1407, system memory 1408 (e.g., RAM)”.
All other limitations in claim 15 are substantially the same as those in claim 7, therefore the same rationale for rejection applies.
Regarding claim 17,
Claim 17 recites a computer-readable medium containing instructions for performing the function of the method of claim 1. Specifically, claim 17 recites A non-transitory computer-readable storage medium, wherein the computer-readable medium stores program code which, when executed by one or more processors of a first device, cause the one or more processors to: [perform the method of claim 1].
Rabinovich recites: (Rabinovich [0080]) “According to one embodiment of the invention, computer system 1400 performs specific operations by processor 1407 executing one or more sequences of one or more instructions contained in system memory 1408. Such instructions may be read into system memory 1408 from another computer readable/usable medium, such as static storage device 1409 or disk drive 1410.”.
All other limitations in claim 17 are substantially the same as those in claim 1, therefore the same rationale for rejection applies.
Regarding claim 18,
Claim 18 recites a chip for performing the function of the method of claim 1. Specifically, claim 18 recites A chip, wherein the chip comprises at least one processor and a data interface, and the at least one processor reads, by using the data interface, instructions stored in a memory, to: [perform the method of claim 1].
Rabinovich recites: (Rabinovich [0079]) “FIG. 7 is a block diagram of an illustrative computing system 1400 suitable for implementing an embodiment of the present invention. Computer system 1400 includes a bus 1406 or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor 1407, system memory 1408 (e.g., RAM)”.
All other limitations in claim 18 are substantially the same as those in claim 1, therefore the same rationale for rejection applies.
Regarding claim 19,
Claim 19 recites a computer-readable medium containing code for performing the function of the method of claim 2. All other limitations in claim 19 are substantially the same as those in claim 2, therefore the same rationale for rejection applies.
Claims 21 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Rabinovich, further in view of Spangenberg.
Regarding claim 21,
Liu and Rabinovich jointly teach The method according to claim 1,
Spangenberg teaches the following further limitations that neither Liu nor Rabinovich teach:
wherein the first device includes a vehicle mounted terminal, ((Spangenberg [0032]) “The vehicle 12 has a camera system 16 with at least one camera 17…image data corresponding to the pictures are generated. The generated image data are transferred from the camera system 16 to a processing unit 22 arranged in the vehicle 12, which unit processes these data”)
and the result of processing the to-be-processed image data or the to-be processed voice data includes different recognized objects in pictures of lanes ((Spangenberg [0033]) “The image data of an object 28 generated by the camera system 16 are processed by the processing unit 22, the image of the object 28 being detected as an object image or as an object, and preferably the object type of the object 28 being classified. In the same manner, traffic signs, lane indicators, street lights, vehicles driving ahead on the lane 14 and oncoming vehicles on an oncoming lane may be detected as objects and their object type may be classified”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Liu, Rabinovich, and Spangenberg by taking the method for neural architecture search including processing and transmission of image data between devices, jointly taught by Liu and Rabinovich, and combining it with the system of Spangenberg where a vehicle’s processing unit processes images including detected objects in lanes, taught by Spangenberg, as Spangenberg teaches: (Spangenberg [0033]) “The comparison of the position of objects 28 in a first picture and a second picture taken after this first picture is for example used to influence the driving behavior and/or to provide the driver of the vehicle 12 with defined information on the environment and on the own vehicle 12”, and providing a driver with information on the environment provides the predictable benefit of assisting them with driving. Such a combination would be obvious.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Benyahia et al. (U.S. Patent Application Publication No. 2020/0104688) teaches a neural architecture search method, including construction of a computational graph with nodes.
Stamoulis et al. “Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours” teaches a neural architecture search method, including single-path / one-level optimization.
Ying et al. “NAS-Bench-101: Towards Reproducible Neural Architecture Search” teaches a neural architecture search method, including additional capabilities for analysis of the search space, such as quantifiable tradeoffs between the use of different operators.
Vasudevan et al. (U.S. Patent Application Publication No. 2023/0252327) teaches methods and systems for convolutional neural network architecture search for image processing tasks.
Jin et al. “Auto-Keras: An Efficient Neural Architecture Search System” teaches a neural architecture search method, including a search strategy that adapts to different GPU memory limits.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights