DETAILED ACTION
This action is in response the communications filed on 04/23/2026 in which claims 1-3, 6-8, 15-17, and 20-23 are amended, and therefore claims 1-26 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/23/2026 has been entered.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 10/28/2025 and 05/01/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claim 16 objected to because of the following informalities:
In claim 16, “select the one or more edges and one or more operations based, at least in part, on one or more that satisfy the one or more comparisons” should be “select the one or more edges and one or more operations based, at least in part, on the one or more comparisons” (the phrase “one or more that satisfy” should be removed)
In claim 16, “performing a search of a search space to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations wherein the selection is based…” should be “performing a search of a search space to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations, wherein the selection is based…” (a comma is missing)
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-26 are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 1, 7, 15 and 21 recite the limitation “the one or more comparisons.” There is insufficient antecedent basis for this limitation in the claim. For examination purposes examiner has interpreted “the one or more comparisons” to be “one or more comparisons.”
Claims 2-6, 8-14, 16-20 and 22-26 are also rejected due to their dependency on a rejected claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
-
Claims 1-26 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Step 1: Claims 1-6 recite one or more processors. Claims 7-14 recite a system comprising processors. Claims 15-20 recite a datacenter comprising processors. Claims 21-26 recite a method. Therefore, claims 1-6 are directed to a manufacture, claims 7-20 are directed to a machine, and claims 21-26 are directed to a process.
With respect to claims 1, 7, 15 and 21:
2A Prong 1: the claim recites a judicial exception.
determine a plurality of candidate connection patterns from the set of candidate edges and the set of candidate operations; (mental process – evaluation or judgement; determine candidate patterns)
compare each of the plurality of candidate connection patterns and one or more constraints, wherein the one or more constraints indicate one or more memory requirements and one or more feasibility criteria; (mental process – evaluation or judgement; compare each of the patters and constraints)
select one or more edges from the set of candidate edges and one or more operations from the set of candidate operations based, at least in part, on the one or more comparisons; and (mental process – evaluation or judgement; select edges and operations based on comparisons)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claim 1) circuitry (claims 7 and 15) one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
obtain a set of candidate edges and a set of candidate operations, wherein the set of candidate edges and the set of candidate operations are usable to generate a plurality of neural networks of a plurality of topologies (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering, obtain edges and operations)
cause an instantiation of one or more neural networks with a topology of the plurality of topologies to perform (claims 1, 7 and 21) an image-based task (claim 15) a medical image segmentation task based, at least in part, on the selected one or more edges and the selected one or more operations (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a neural network to perform a task based on the selected edges and operations)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 1) circuitry (claims 7 and 15) one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
obtain a set of candidate edges and a set of candidate operations, wherein the set of candidate edges and the set of candidate operations are usable to generate a plurality of neural networks of a plurality of topologies (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering, obtain edges and operations, and WURC: receiving or transmitting data over a network– MPEP 2106.05(d)(II)(i))
cause an instantiation of one or more neural networks with a topology of the plurality of topologies to perform (claims 1, 7 and 21) an image-based task (claim 15) a medical image segmentation task based, at least in part, on the selected one or more edges and the selected one or more operations (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a neural network to perform a task based on the selected edges and operations)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 2, 8, 16 and 22:
2A Prong 1: the claim recites a judicial exception.
(claim 2) perform a search to select the one or more edges and one or more operations based, at least in part, on the one or more comparisons
(claim 8) perform a first search to select the one or more edges and one or more operations based, at least in part, on the one or more comparisons
(claim 16) perform a search to select the one or more edges and one or more operations based, at least in part, on one or more that satisfy the one or more comparisons
(claim 22) selecting a first set of one or more operations and a first set of one or more edges for the one or more neural networks based, at least in part, on the one or more comparisons
(mental process – evaluation or judgement; select edges and operations based on comparisons)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claim 2) the circuitry (claims 8 and 16) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 2) the circuitry (claims 8 and 16) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 3 and 17:
2A Prong 1: the claim recites a judicial exception.
(claim 3) cause one or more second edges and one or more second operations to be selected from at least one of the set of candidate edges and the set of candidate operations or from a second set of candidate edges and set of candidate operation;
wherein the one or more second edges and one or more second operations are different from the one or more edges and one or more operations; and
wherein the one or more second edges and one or more second operations satisfy one or more second memory constraints that are different from a first constraint satisfied by the one or more edges and one or more operations
(claim 17) cause one or more second edges and one or more second operations to be selected, based on one or more second memory constraints, from at least one of the set of candidate edges and the set of candidate operations or from a different set of candidate edges and set of candidate operations; and
wherein the one or more second memory constraints are different from the one or more constraints
(mental process – evaluation or judgement, select second edges and operations, which are different from the (first) edges and operations and satisfy second memory constraints)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claim 3) the circuitry (claim 17) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 3) the circuitry (claim 17) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 4:
2A Prong 1: the claim recites a judicial exception.
perform a search to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations in accordance with a set of one or more search parameters determined at least in part on an amount of memory to be used by the one or more neural networks (mental process – evaluation or judgement, select edges and operations according to search parameters)
2A Prong 2: This judicial exception is not integrated into a practical application.
• the circuity (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
• the circuity (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 5:
2A Prong 1: the claim recites a judicial exception.
wherein a percentage of a maximum memory usage of operations associated with one or more candidate feature nodes of a search space comprising the plurality of neural networks is less than or equal to an amount of memory (mental process – evaluation or judgement, a part of a memory usage of operations is less than an amount of memory)
With respect to claims 6, 11, 18 and 24:
2A Prong 1: the claim recites a judicial exception.
(claim 6) cause the one or more edges and one or more operations to be selected by performing a joint two-level search of a topology search space and a cell search space
(claim 11) cause the one or more edges and one or more operations to be selected by performing a joint two-level search of a topology search space and a cell search space to identify the one or more neural networks for an image-based task
(claim 18) perform a joint two-level search of a topology search space and a cell search space to identify the one or more neural networks for the medical image segmentation task
(claim 24) performing a joint two- level search of a topology search space and a cell search space to identify the one or more neural networks for an image-based task
(mental process – evaluation or judgement, select/identify neural networks (with edges and operations) using a joint search of a topology search and a cell search)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claim 6) the circuitry (claims 11 and 18) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 6) the circuitry (claims 11 and 18) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 9, 14, 19 and 23:
2A Prong 1: the claim recites a judicial exception.
(claim 9) select a connection pattern, from a plurality of candidate connection patterns between a first layer and a second layer of the one or more neural networks with a topology of the plurality of topologies, based at least in part on probabilities of each of the plurality of candidate connection patterns
(claim 14) select a connection pattern, from a plurality of candidate connection patterns between layers of the plurality of neural networks of a plurality of topologies, based at least in part on probabilities of the plurality of candidate connection patterns
(claim 19) perform a search of a search space to cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected, wherein the search comprises selecting a connection pattern between layers of the one or more neural networks, from a plurality of candidate connection patterns, based at least in part on probabilities of the plurality of candidate connection patterns
(claim 23) performing a search of a search space to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations wherein the selection is based at least in part on probabilities of the plurality of candidate connection patterns
(mental process – evaluation or judgement, select edges and operations, and select a connection pattern based on probabilities of the connection patterns)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claims 9, 14 and 19) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claims 9, 14 and 19) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 10:
2A Prong 1: the claim recites a judicial exception.
select a feature node from a set of candidate features nodes for one or more layers of the one or more neural networks with a topology of the plurality of topologies, wherein the set of candidate feature nodes comprises feature nodes at different image scales that comprise a plurality of candidate edges that connect to a feature node in a previous layer (mental process – evaluation or judgement, select a node from a set of nodes with different scales)
2A Prong 2: This judicial exception is not integrated into a practical application.
the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 12 and 25:
2A Prong 2: This judicial exception is not integrated into a practical application.
wherein the one or more neural networks are to perform an image segmentation task (a particular technological environment or field of use and – MPEP 2106.05(h))
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
• wherein the one or more neural networks are to perform an image segmentation task (a particular technological environment or field of use and – MPEP 2106.05(h)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 13 and 20:
2A Prong 1: the claim recites a judicial exception.
(claim 13) cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected by performing a search of a topology search space comprising a plurality of candidate edges that connect candidate feature nodes of a plurality of layers and a cell search space comprising a plurality of candidate operations
(claim 20) perform a search of a search space to cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected, wherein the search comprises selecting a connection pattern from a feasible set of candidate connection patterns between layers of the one or more neural networks, wherein each feasible connection pattern in the feasible set of candidate connection patterns comprises valid input connections and output connections between the layers
(mental process – evaluation or judgement, select edges that connect nodes or feasible patterns and operations)
2A Prong 2: This judicial exception is not integrated into a practical application.
(claims 19-20) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claims 19-20) the one or more processors (mere instructions to apply an exception - MPEP 2106.05(f), (2) invoking general computers as a tool to perform a process)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 26:
2A Prong 1: the claim recites a judicial exception.
• performing a search of a multi-scale topology search space by converting the multi-scale topology search space into a sequential search space comprising a super node for each respective layer of a plurality of layers, wherein each super node comprises a set of candidate feature nodes at the respective layer (mental process – evaluation or judgement, converting the multi-scale search to a sequential search with a super node for each layer and feature nodes at the layer)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6-8, 11-13, 15-17, 18, 20-22 and 24-25 rejected under 35 U.S.C. 103 as being unpatentable over Yu ("C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation" 20200420) in further view of Cassimon ("Designing resource-constrained neural networks using neural architecture search targeting embedded devices" 20200531)
In regard to claims 1, 7, 15 and 21, Yu teaches: One or more processors comprising, circuitry to: (Yu, p. 7 "The coarse search stage takes 5 days with 64 NVIDIA V100 GPUs with 16GB memory. In fine stage, the super network training costs 10 hours with 8 GPUs...")
PNG
media_image1.png
176
1508
media_image1.png
Greyscale
obtain a set of candidate edges and a set of candidate operations, wherein the set of candidate edges and the set of candidate operations are usable to generate a plurality of neural networks of a plurality of topologies; (Yu, p. 2 "Figure 2... Each path from the left-most node to the right-most node is a candidate architecture. Each color represents one category of operations, e.g. depthwise conv, dilated conv, or 2D/3D/P3D conv which are more common in medical image area... The macro-level topology is determined by coarse stage search, while the micro-level operations are further selected in fine stage search."; p. 3 "we develop a coarse-to-fine neural architecture search method for automatically designing 3D segmentation networks [generate neural networks of topologies]… the architecture search space A consists of topology search space S, [all the paths in each network topology, obtain a set of candidate edges] which is represented by a directed acyclic graph (DAG), and cell operation space C, [obtain a set of candidate operations] which is represented by the color of each node in the DAG. ")
determine a plurality of candidate connection patterns from the set of candidate edges and the set of candidate operations; (Yu, p. 3 "the architecture search space A consists of topology search space S, which is represented by a directed acyclic graph (DAG), and cell operation space C, which is represented by the color of each node in the DAG. Each network candidate is a sub-graph s ∈ S with color scheme c ∈ C... [determining candidate connection patters from S and C]")
… cause an instantiation of one or more neural networks with a topology of the plurality of topologies to (claims 1, 7 and 21) an image-based task (claim 15) a medical image segmentation task based, at least in part, on the selected one or more edges and the selected one or more operations. (Yu, p. 2 "we propose a coarse-to-fine neural architecture search scheme for 3D medical image segmentation [an image-based task, a medical image segmentation task] (see Fig. 2)."; p. 3 "we develop a coarse-to-fine neural architecture search method for automatically designing 3D segmentation networks [an instantiation of neural networks of topologies]"; p. 7 "The final network architecture based on the topology searched in coarse stage and operations searched in fine stage [based on selected edges and operations] is shown in Fig. 5.")
Yu does not teach, but Cassimon teaches: compare each of the plurality of candidate connection patterns and one or more constraints, wherein the one or more constraints indicate one or more memory requirements and one or more feasibility criteria; (Cassimon, p. 1, Abstract "In this paper, we present an improvement on a recent NAS method, Efficient Neural Architecture Search (ENAS). We adapt ENAS to not only take into account the network’s performance, but also various constraints that would allow these networks to be ported to embedded devices"; p. 2-3, 3. Methods "For our research we intend to improve on ENAS by introducing extra constraints to the reward function of the controller... The first is a set of hard constraints, and the second is a set of soft constraints... In order to implement these constraints, we use Equation 1 to determine the reward, R given to our controller during its training. R(μ, Ch, Cs) = ... (1)... we have k hard constraints... and n soft constraints... The first hard constraint we used is the amount of memory the model uses. [memory requirements] Determining the memory usage requires us to traverse the cell’s graph, [connection patterns] and determine the size of the block’s inputs based on the size of the outputs leading into this block... We now formalize our hard memory constraints as: C_h, memory = 0 if S_model + ε ≥ S_device 1 if S_model + ε < S_device (2) [comparing with memeory requirements/constraints]... The second hard constraint we introduced is inference latency... we consider the constraint to be violated. This constraint can be formalized as shown in Equation 3: C_h, complexity = 0 if Latency_model + ε ≥ Latency_max 1 if Latency_model + ε < Latency_max (3) [comparing with feasibility criteria]"; the reward is based on all the constraints or comparisons)
select one or more edges from the set of candidate edges and one or more operations from the set of candidate operations based, at least in part, on the one or more comparisons; and (Cassimon, p. 5, 4.2.2. Configuration "We organize our results based on whether or not our constraints were enabled [based on the constraints/comparisons]... In our experiments, our controller is allowed to choose a cell size in the range [2-24] [selecting edges]. Our search space consists of four activation functions: identity, sigmoid, ReLU and tanh. [selecting operations]")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Yu to incorporate the teachings of Cassimon by including neural architecture search with various constraints. Doing so would allow the system to perform tasks efficiently on an embedded device. (Cassimon, p. 1, Abstract "We adapt ENAS to not only take into account the network’s performance, but also various constraints that would allow these networks to be ported to embedded devices... In order to show the efficacy of our system... capable of classifying 32x32 RGB images at a rate of 1 FPS on an embedded device.")
Claims 7, 15 and 21 recite substantially the same limitation as claim 1, therefore the rejection applied to claim 1 also apply to claims 7, 15 and 21. In addition, Yu teaches: one or more processors (Yu, p. 7 "The coarse search stage takes 5 days with 64 NVIDIA V100 GPUs with 16GB memory. In fine stage, the super network training costs 10 hours with 8 GPUs...")
In regard to claims 2, 8, 16 and 22, Yu does not teach, but Cassimon teaches:
(claim 2) wherein the circuitry is further to perform a search to select the one or more edges and one or more operations based, at least in part, on the one or more comparisons
(claim 8) wherein the one or more processors are further to perform a first search to select the one or more edges and one or more operations based, at least in part, on the one or more comparisons
(claim 16) wherein the one or more processors are further to perform a search to select the one or more edges and one or more operations based, at least in part, on one or more that satisfy the one or more comparisons
(claim 22) further comprising selecting a first set of one or more operations and a first set of one or more edges for the one or more neural networks based, at least in part, on the one or more comparisons
(Cassimon, p. 5, 4.2.2. Configuration "We organize our results based on whether or not our constraints were enabled [based on the constraints/comparisons]... In our experiments, our controller is allowed to choose a cell size in the range [2-24] [selecting edges]. Our search space consists of four activation functions: identity, sigmoid, ReLU and tanh. [selecting operations]")
The rationale for combining the teachings of Yu and Cassimon is the same as set forth in the rejection of claim 1.
In regard to claims 3 and 17, Yu does not teach, but Cassimon teaches:
(claim 3) wherein the circuitry is further to cause one or more second edges and one or more second operations to be selected from at least one of the set of candidate edges and the set of candidate operations or from a second set of candidate edges and set of candidate operation;
wherein the one or more second edges and one or more second operations are different from the one or more edges and one or more operations; and
wherein the one or more second edges and one or more second operations satisfy one or more second memory constraints that are different from a first constraint satisfied by the one or more edges and one or more operations
(claim 17) wherein the one or more processors are further to cause one or more second edges and one or more second operations to be selected, based on one or more second memory constraints, from at least one of the set of candidate edges and the set of candidate operations or from a different set of candidate edges and set of candidate operations; and
wherein the one or more second memory constraints are different from the one or more constraints
(Cassimon, p. 1 Abstract "…various constraints that would allow these networks to be ported to embedded devices."; p. 5, 4.2.2. Configuration "We organize our results based on whether or not our constraints were enabled [based on (second) constraints/comparisons]... In our experiments, our controller is allowed to choose a cell size in the range [2-24] [selecting (second) edges]. Our search space consists of four activation functions: identity, sigmoid, ReLU and tanh. [selecting (second) operations]"; Neural architecture search in Cassimon can be used to build a different model for a different embedded device with a different memory constraint, i.e. various models for various embedded devices with various constraints)
The rationale for combining the teachings of Yu and Cassimon is the same as set forth in the rejection of claim 1.
In regard to claims 6, 11, 18 and 24, Yu teaches:
(claim 6) wherein the circuitry is further to cause the one or more edges and one or more operations to be selected by performing a joint two-level search of a topology search space and a cell search space
(claim 11) wherein the one or more processors further cause the one or more edges and one or more operations to be selected by performing a joint two-level search of a topology search space and a cell search space to identify the one or more neural networks for an image-based task.
(claim 18) wherein the one or more processors are to perform a joint two-level search of a topology search space and a cell search space to identify the one or more neural networks for the medical image segmentation task.
(claim 24) further comprising performing a joint two- level search of a topology search space and a cell search space to identify the one or more neural networks for an image-based task.
(Yu, p. 2 "The macro-level topology is determined by coarse stage search, while the micro-level operations are further selected in fine stage search. [a joint two-level search of a topology search space and a cell search space]"; p. 3 "we develop a coarse-to-fine neural architecture search method for automatically designing 3D segmentation networks"; p. 5 "Therefore, the final network architecture N(s*; c*; w) is constructed.")
In regard to claims 12 and 25, Yu teaches: wherein the one or more neural networks are to perform an image segmentation task. (Yu, p. 2 "we propose a coarse-to-fine neural architecture search scheme for 3D medical image segmentation [a medical image segmentation task] (see Fig. 2).")
In regard to claims 13 and 20, Yu teaches:
(claim 13) wherein the one or more processors further cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected by performing a search of a topology search space comprising a plurality of candidate edges that connect candidate feature nodes of a plurality of layers and a cell search space comprising a plurality of candidate operations.
(claim 20) wherein the one or more processors are to perform a search of a search space to cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected, wherein the search comprises selecting a connection pattern from a feasible set of candidate connection patterns between layers of the one or more neural networks, wherein each feasible connection pattern in the feasible set of candidate connection patterns comprises valid input connections and output connections between the layers.
(Yu, p. 4 3.3. Coarse Stage: Macrolevel Search "An illustration of how the priors help prune search space is shown in Fig. 3. Therefore, the search space S is pruned to S' [select edges from the set of edges S] ... S' = PriorPrune(s), (4)"; p. 5, 3.4. Fine Stage: Microlevel Search "The set of possible operations, O, [select operations from the set of operations C or O] consisting of the following 3 choices: (1) 3x3x3 3D convolution; (2) 3x3x1 followed by... "; p. 2 "Figure 2... Each path from the left-most node to the right-most node [edges that connect nodes of layers] is a candidate architecture"; p. 4 "Figure 3... The grey nodes are eliminated entirely from the graph. Besides, many illegal paths have been pruned off as well. An example of illegal path and legal path [a feasible set of connection patterns, valid input and output connections] is shown as the orange line path and green line path separately."; Fig. 3 pruning paths in the coarse stage [select edges], and Fig. 5 operations searched in fine stage [select operations])
Claims 4-5 rejected under 35 U.S.C. 103 as being unpatentable over Yu and Cassimon, as applied to claim 1 and 15, and in further view of Fedorov ("SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers" 20190528)
In regard to claim 4, Yu teaches: wherein the circuitry is further to perform a search to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations (Yu, p. 4 3.3. Coarse Stage: Macrolevel Search "An illustration of how the priors help prune search space is shown in Fig. 3. Therefore, the search space S is pruned to S' [select edges from the set of edges S] ... S' = PriorPrune(s), (4)"; p. 5, 3.4. Fine Stage: Microlevel Search "The set of possible operations, O, [select operations from the set of operations C or O] consisting of the following 3 choices: (1) 3x3x3 3D convolution; (2) 3x3x1 followed by... "; Fig. 3 pruning paths in the coarse stage [select edges], and Fig. 5 operations searched in fine stage [select operations])
Yu and Cassimon do not teach, but Fedorov teaches: in accordance with a set of one or more search parameters determined at least in part on an amount of memory to be used by the one or more neural networks. (Fedorov, p. 4 "Our search space is designed to encompass CNNs of varying depth, width, and connectivity. [selecting edges and operations] Each graph consists of optional input downsampling followed by a variable number of blocks... "; p.5 "Pruning [37] is essential to MCU deployment using SpArSe, as it heavily reduces the model size and working memory without significantly impacting classification accuracy"; p. 6 "Because our search space includes such a diversity of parameters [in accordance with a set of one or more search parameters] , including architectural parameters, pruning hyperparameters, etc., we find it helpful to perform the search in stages..."; p. 2 "C2 : The model parameters must not exceed the ROM (flash memory) capacity. [the amount of memory]"; p. 7 "we address C2 by showing that SpArSe finds CNNs with higher accuracy and fewer parameters than previously published methods"; p. 8 "The results show that including pruning as part of the optimization yields roughly an 80x reduction in number of parameters...")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Yu and Cassimon to incorporate the teachings of Fedorov by including automatically design CNNs implemented on multiple microcontroller units (MCUs). Doing so would make the CNNs being small enough to meet the strict MCU working memory constraint. (Fedorov, p. 1 "The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment… The CNNs we find are more accurate and up to 7.4x smaller than previous approaches, while meeting the strict MCU working memory constraint.")
In regard to claim 5, Yu and Cassimon do not teach, but Fedorov teaches: wherein a percentage of a maximum memory usage of operations associated with one or more candidate feature nodes of a search space comprising the plurality of neural networks is less than or equal to an amount of memory. (Fedorov, p. 4 "MODELSIZE(ω), or MS, is the number of bits needed to store the model parameters ω, WORKINGMEMORY_l(Ω) is the working memory in bits needed to compute the output of layer l, with the maximum taken over the L layers to account for in-place operations. [a maximum memory usage of operations associated with candidate feature nodes]"; p. 8 "Table 3: Comparison of Bonsai with SpArSe for WM model (5). The first row shows the highest accuracy model for WM ≤ 2KB [less than or equal to an amount of memory] and the second row shows the highest accuracy model for WM, MS ≤ 2KB."; part of a memory usage of operations is less than an amount of memory)
The rationale for combining the teachings of Yu, Cassimon and Fedorov is the same as set forth in the rejection of claim 4.
Claims 9-10, 14, 19 and 23 rejected under 35 U.S.C. 103 as being unpatentable over Yu and Cassimon, as applied to claims 7, 15 and 21, and in further view of Liu ("Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation" 20190406)
In regard to claims 9, 14, 19 and 23, Yu and Cassimon do not teach, but Liu teaches:
(claim 9) wherein the one or more processors are further to select a connection pattern, from a plurality of candidate connection patterns between a first layer and a second layer of the one or more neural networks with a topology of the plurality of topologies, based at least in part on probabilities of each of the plurality of candidate connection patterns.
(claim 14) wherein the one or more processors are further to select a connection pattern, from a plurality of candidate connection patterns between layers of the plurality of neural networks of a plurality of topologies, based at least in part on probabilities of the plurality of candidate connection patterns.
(claim 19) wherein the one or more processors are further to perform a search of a search space to cause the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations to be selected, wherein the search comprises selecting a connection pattern between layers of the one or more neural networks, from a plurality of candidate connection patterns, based at least in part on probabilities of the plurality of candidate connection patterns.
(claim 23) further comprising performing a search of a search space to select the one or more edges from the set of candidate edges and one or more operations from the set of candidate operations wherein the selection is based at least in part on probabilities of the plurality of candidate connection patterns
(Liu, p. 86 "the β values can be interpreted as the 'transition probability' between different 'states' (spatial resolution) across different 'time steps' (layer number)... our goal is to find the path with the 'maximum probability' from start to end. [select a connection pattern between layers based on probabilities]"; p. 84, 3.2. Network Level Search Space "We illustrate our network level search space in Fig. 1. Our goal is then to find a good path [select edges] in this L-layer trellis."; p. 84, 3.1. Cell Level Search Space "The set of possible layer types, O, [select operations] consists of the following 8 operators, all prevalent in modern CNNs: 3 × 3 depthwise-separable conv 5 × 5 depthwise-separable conv 3 × 3 atrous conv with rate 2 5 × 5 atrous conv with rate 2 3 × 3 average pooling..."; all the transition probabilities across resolutions and layers are candidate connection patterns)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Yu and Cassimon to incorporate the teachings of Liu by including a hierarchical architecture search space with probabilities for choosing paths. Doing so would achieve state-of-the-art performance specifically for semantic image segmentation. (Liu, P. 82 "we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space... We demonstrate the effectiveness of the proposed method...specifically for semantic image segmentation, attains state-of-the-art performance...")
In regard to claim 10, Yu and Cassimon do not teach, but Liu teaches: wherein the one or more processors are further to select a feature node from a set of candidate features nodes for one or more layers of the one or more neural networks with a topology of the plurality of topologies, wherein the set of candidate feature nodes comprises feature nodes at different image scales that comprise a plurality of candidate edges that connect to a feature node in a previous layer. (Liu, p. 86 "each layer l will have at most 4 hidden states {4Hl, 8Hl, 16Hl, 32Hl} [feature nodes at different image scales], with the upper left superscript indicating the spatial resolution."; p. 4 "Figure 1… a path along the blue nodes [a set of candidate features nodes for layers 1..L] represents a candidate network level architecture")
PNG
media_image2.png
266
638
media_image2.png
Greyscale
The rationale for combining the teachings of Yu, Cassimon and Liu is the same as set forth in the rejection of claim 9.
Claim 26 rejected under 35 U.S.C. 103 as being unpatentable over Yu and Cassimon, as applied to claim 21, and in further view of Garg ("Revisiting Neural Architecture Search")
PNG
media_image3.png
336
318
media_image3.png
Greyscale
In regard to claim 26, Yu and Cassimon do not teach, but Garg teaches: further comprising performing a search of a multi-scale topology search space by converting the multi-scale topology search space into a sequential search space (Garg, p. 4 "The architecture search is based upon the differentiable architecture search (Liu et al. (2019); Cai et al. (2019)). In this method, we start with an over-parameterized (parent) network having all operations in the search space [converting the multi-scale topology search space] (e.g., convolution, pooling, etc.)."; p. 4 Algorithm 1 ReNAS "for each node v in G do 1. Partition the node into K channel blocks..."; p. 5 "The procedure starts by constructing the parent network by wirin 3.1. Every node’s channels in are partitioned into channel blocks... This process is repeated [a sequential search] for every node of every DAG in the over-parameterized network until convergence.") comprising a super node for each respective layer of a plurality of layers, wherein each super node comprises a set of candidate feature nodes at the respective layer. (Garg, p. 5 "Figure 2 depicts the node level expansion. The operation sampling is based on the path sampling heuristic of (Cai et al. (2019)), in which two candidate paths (operations) are sampled from a multinomial distribution over all operations."; see Figure 2, Nodes u, v, and w are [super nodes for respective layers], and those operation 3x3, 5x5, etc. are [feature nodes at respective layers])
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Yu and Cassimon to incorporate the teachings of Garg by including a repeated for the connections and operations. Doing so would balance the exploration and exploitation of the search space. (Garg, p. 1 "Our method starts from a complete graph mapped to a neural network and searches for the connections and operations by balancing the exploration and exploitation of the search space. The results are on-par with the SOTA performance with methods that leverage handcrafted blocks.")
Response to Arguments
Applicant's amendments with respect to the claim objections have been fully considered, but objections are maintained because of the informalities in the new limitations.
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 101 have been fully considered but they are not persuasive:
Applicant argues: (p. 11-12) Step 2A Prong 1: Claim 1 Does Not Recite a Judicial Exception… By way of non-limiting example, amended Claim 1 recites, inter alia, the following recitations that at least cannot be properly interpreted as reciting any of the Enumerated Groupings.… cause an instantiation of one or more neural networks…
Examiner answers: The limitation “cause an instantiation of one or more neural networks with a topology of the plurality of topologies to perform (claims 1, 7 and 21) an image-based task (claim 15) a medical image segmentation task based, at least in part, on the selected one or more edges and the selected one or more operations” is not an abstract idea. Instead, the “cause” step is an additional element evaluated under step2A Prong 2 and step 2B. See below for details.
Applicant argues: (p. 12-13) Step 2A Prong 2: Any Alleged Judicial Exception Is Integrated Into A
Practical… cause an instantiation of one or more neural networks… As previously indicated, none of these recitations recite "mental processes." The claim, when read as a whole, integrates the alleged abstract idea into a practical application by including specific recitations that meaningfully limit its use and result in improved computer-implemented functionality for instantiation of neural networks with selected topologies to perform image-based tasks…
Examiner answers: The limitation “cause an instantiation of one or more neural networks with a topology of the plurality of topologies to perform (claims 1, 7 and 21) an image-based task (claim 15) a medical image segmentation task based, at least in part, on the selected one or more edges and the selected one or more operations” is ‘mere instructions to apply an exception’ – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; using a neural network to perform a task based on the selected edges and operations.
Further, the claim does not recite any other additional elements that are indicative of integration into a practical application. The “determine, compare, and select” steps are abstract ideas, and therefore cannot provide an improvement. (See MPEP 2106.05(a): “It is important to note, the judicial exception alone cannot provide the improvement.”)
Applicant argues: (p. 13-14) Step 2B: The Additional Claim Recitations Amount To Significantly More Than The Judicial Exception… Here, Claim 1 recites a specific, machine-implemented neural architecture search technique in which circuitry obtains… selects… causing… they define a particular technological implementation for instantiating neural networks for image based tasks under constraints.
Examiner answers: The “determine, compare, and select” steps are abstract ideas. The “obtain and cause” steps are ‘insignificant extra-solution activity’ – MPEP 2106.05(g) and ‘mere instructions to apply an exception’ – MPEP 2106.05(f), respectively. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea.
Applicant argues: (p. 14-19) Independent Claim 7… Independent Claim 15… Independent Claim 21
Examiner answers: The arguments presented here for independent claims 7, 15 and 21 apply equally to claim 1. See the examiner's responses above for details.
Applicant's arguments with respect to the rejection of the claims under 35 U.S.C. 102/103 have been fully considered but they are moot:
Applicant argues: (p. 20-21) A. Independent Claim 1… but Yu does not disclose the claimed selection process based on candidate connection patterns that are compared against both memory requirements and feasibility criteria. Additionally, Yu does not teach or suggest that the constraints "indicate one or more memory requirements and one or more feasibility criteria.”… Independent Claims 7, 15, and 21
Examiner answers: the arguments do not apply to the references (Cassimon) being used in the current rejection. Yu teaches the “obtain, determine, and cause” steps. Cassimon teaches the “compare and select” steps, and Cassimon teaches the selection process is based on memory requirements and feasibility criteria.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SU-TING CHUANG whose telephone number is (408)918-7519. The examiner can normally be reached Monday - Thursday 8-5 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew J. Jung can be reached on (571) 270-3779. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.C./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights