Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is sent in response to Application’s Communication received on 01/16/2024 for application number 18/414068. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawing, Abstract, Oath/Declaration, and Claims.
Claims (1-9) and (10-18) are presented for examination.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 10/07/2025, 01/08/2025, 05/13/2024, 01/16/2024 were filed prior to current Office Action. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement were being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 9, 10, 12-13 and 18 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Tran et al. US Patent Application Publication US 20200143227 A1 (hereinafter Tran) in view of Zehngut et al. US Patent Application Publication US 20220147680 A1 (hereinafter Zehngut) and further in view of Wen et al. US Patent Application Publication US 20230409867 A1 (hereinafter Wen).
Regarding claim 1, Tan teaches a method for selecting an artificial intelligence (AI) model in neural architecture search (NAS), the method comprising (Abstract, wherein Tran teaches an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models)
Tran teaches measuring a scale of receptive field for a plurality of neural network layers corresponding to each of a plurality of candidate AI models (FIG. 3, [0069], [0090-0092] wherein Tran describes measuring receptive fields and measuring by number of multiply-adds for depthwise separable convolution with kernel 5×5 and 3×3).
Tran does not teach plurality of candidate AI models.
However in analogous of selecting an artificial intelligence model in neural architecture search, Zehngut teaches plurality of candidate AI models ([0017], [0036-0037], [0050], [0054], wherein Zehngut describes a controller based NAS methods traverse the search space by individually considering neural network candidates (e.g., candidates for the neural network 400), where each candidate is trained and evaluated. As a result, controller-based NAS methods may be performed on a general search space with minimal limitations. In some examples, controller-based NAS methods are trained such that the output is determined to be within a threshold of a desired output).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Tan with Zehngut by incorporating the method of plurality of candidate AI models of Zehngut into the method of a method for selecting an artificial intelligence (AI) model in neural architecture search (NAS) of Tan for the purpose of modifying the filters so that they activate when they detect a particular feature within the input. (Zehngut: [0037]).
Tran does not teach determining a first score for a first group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the first group of neural network layers, the scale of the receptive field for each of the first group of neural network layers being smaller than a size of an object; determining a second score for a second group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the second group of neural network layers, the scale of the receptive field for each of the second group of neural network layers being greater than the size of the object; determining a third score for each of the plurality of candidate AI models as a function of the first score and the second score; and selecting, based on the third score, a candidate AI model among the plurality of candidate AI models for training and deployment, the candidate AI model having a highest third score among the third scores of the plurality of candidate AI models.
However in analogous of selecting an artificial intelligence model in neural architecture search, Wen teaches determining a first score for a first group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the first group of neural network layers, the scale of the receptive field for each of the first group of neural network layers being smaller than a size of an object; determining a second score for a second group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the second group of neural network layers, the scale of the receptive field for each of the second group of neural network layers being greater than the size of the object; determining a third score for each of the plurality of candidate AI models as a function of the first score and the second score (FIGS. 1-3, Abstract, Claims 1-2, 11-12, 19-20 text, [0001-0013], [0019-0025], [0039-0063] wherein Wen as illustrated in FIGS. 2-3, incorporates NAS models for multi-task-dense predictions and sample from a search space that includes network architecture components, these neural network architecture components may include, for instance, different types of neural network layers and/or layers having different parameters. These sampled neural network architecture components may be assembled into a candidate MT-DP architecture. Wherein Wen obtains sets of tasks and scales various dense predictions about field as illustrated in FIG. 1, wherein the scaling involves different metrics that represent different scores for the sets of tasks and select and deploy one or more of the candidate MT-DP architectures) and selecting, based on the third score, a candidate AI model among the plurality of candidate AI models for training and deployment, the candidate AI model having a highest third score among the third scores of the plurality of candidate AI models ([0006], [0025] wherein Wen describes a method may further include selecting and deploying, on the edge computing system, one or more of the candidate MT-DP architectures based on one or more of the performance metrics)
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Tan with Wen by incorporating the method of determining a first score for a first group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the first group of neural network layers, the scale of the receptive field for each of the first group of neural network layers being smaller than a size of an object; determining a second score for a second group of neural network layers among the plurality of neural network layers based on the scale of the receptive field for the second group of neural network layers, the scale of the receptive field for each of the second group of neural network layers being greater than the size of the object; determining a third score for each of the plurality of candidate AI models as a function of the first score and the second score; and selecting, based on the third score, a candidate AI model among the plurality of candidate AI models for training and deployment, the candidate AI model having a highest third score among the third scores of the plurality of candidate AI models of Tan for the purpose of incorporating a NAS module that may sample one or more candidate MT-DP architectures from a search space of neural network architecture components. (Wen: [0063]).
Regarding claim 3, Tran as modified by Zehngut and Wen teach wherein the determining the first score comprises: determining a first weightage value associated with each layer of the first group of neural network layers based on the scale of the receptive field for the first group of neural network layers; and determining the first score based on the first weightage value ([0016], [0035], [0139] wherein Zehngut describes a neural networks that is generally comprised of an architecture of layers and network weights, wherein the neural networks is a form of data processing, where input data or input signals are processed with a set of operations and grouped into layers and wherein each layer is parameterized by a set of operations, an order of the operations, and coefficients of the operations (e.g., which may be referred to as network weights)). ([0008-0009], [0023] wherein Wen describes the neural architecture components in the search space that may include, for instance, neural network layers having various layer parameters. One layer parameter may be a layer type. A layer type may include, for instance, an inverted bottleneck (IBN) layer, a fused IBN layer, etc. In addition to a layer type, each neural network layer may have any number of other per-layer parameters, including but not limited to kernel size, output channel multipliers (e.g., {0.5, 0.75, 1.0, 1.5}), stride, and expansion ratios).
Regarding claim 4, Tran as modified by Zehngut and Wen teach wherein the determining the second score comprises: determining a second weightage value associated with each layer of the second group of neural network layers based on the scale of the receptive field for the second group of neural network layers; and determining the second score based on the second weightage value ([0016], [0035], [0139] wherein Zehngut describes a neural networks that is generally comprised of an architecture of layers and network weights, wherein the neural networks is a form of data processing, where input data or input signals are processed with a set of operations and grouped into layers and wherein each layer is parameterized by a set of operations, an order of the operations, and coefficients of the operations (e.g., which may be referred to as network weights)). ([0008-0009], [0023] wherein Wen describes the neural architecture components in the search space that may include, for instance, neural network layers having various layer parameters. One layer parameter may be a layer type. A layer type may include, for instance, an inverted bottleneck (IBN) layer, a fused IBN layer, etc. In addition to a layer type, each neural network layer may have any number of other per-layer parameters, including but not limited to kernel size, output channel multipliers (e.g., {0.5, 0.75, 1.0, 1.5}), stride, and expansion ratios).
Regarding claim 9, Tran as modified by Zehngut and Wen teach wherein the plurality of candidate AI models are generated by an NAS controller (Claim 1 text, [0005-0009], [0022], [0025], [0039], [0043], [0047], [0056-0065] wherein Wen teaches NAS controller that generates candidates).
Claim 10 is similar in scope to claim 1 therefore the claims are rejected under similar rationale.
Claim 12 is similar in scope to claim 3 therefore the claims are rejected under similar rationale.
Claim 13 is similar in scope to claim 4 therefore the claims are rejected under similar rationale.
Claim 18 is similar in scope to claim 9 therefore the claims are rejected under similar rationale.
Claims 2, 7, 11 and 16 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Tran et al. US Patent Application Publication US 20200143227 A1 (hereinafter Tran) in view of Zehngut et al. US Patent Application Publication US 20220147680 A1 (hereinafter Zehngut) and further in view of Wen et al. US Patent Application Publication US 20230409867 A1 (hereinafter Wen) further in view of Ivan et al. US Patent Application US 12462549 A1 (hereinafter Ivan).
Regarding claim 2, Tran, Zehngut and Wen do not teach wherein the second group of neural network layers have a depth that is greater than a depth of the first group of neural network layers.
However in analogous of selecting an artificial intelligence model in neural architecture search, Ivan teaches wherein the second group of neural network layers have a depth that is greater than a depth of the first group of neural network layers (¶ 39, ¶ 52 wherein Ivan teaches a feasible encoder architecture candidates that are automatically or semi-automatically constructed by considering the predefined runtime limit by using multiple building blocks. A further restriction for computational hardware, specifically embedded hardware, is the availability of certain neural network layers. Many SoCs (SoC: system on a chip) support only a small subset of layers and the list of available layers varies greatly depending on the specific hardware. Therefore, depending on the specific hardware, a set of building blocks of the encoders is predefined, wherein encoder layer includes a certain receptive field. The receptive field of each encoder layer is indicated by the number arranged below the circle representing the respective encoder layer. The receptive field of an encoder layer increases with the depth of encoder, i.e. an encoder layer arranged in a greater depth comprises a higher receptive field as an encoder layer arranged in a lower depth).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Ivan with Tran, Zehngut and Wen by incorporating the method of wherein the second group of neural network layers have a depth that is greater than a depth of the first group of neural network layers of Ivan into the method for selecting an artificial intelligence (AI) model in neural architecture search (NAS) of Tran, Zehngut and Wen for the purpose of incorporating receptive field of each encoder layer is indicated by the number arranged below the circle representing the respective encoder layer. (Ivan: ¶ 52).
Regarding claim 7, Tran as modified by Zehngut, Wen and Ivan teach wherein the measuring the scale of the receptive field for the plurality of neural network layers comprises: adding information including the scale of the receptive field of the candidate AI model in a feature map (Abstract, Claim 1 text, ¶ 9-10, ¶ 16, ¶ 26, ¶ 38, ¶ 41, wherein Ivan teaches a method for determining an encoder architecture of a convolutional neural network configured to process image processing tasks. For each image processing task), characteristic scale distribution is calculated based on training data. Encoder architecture candidates are generated, each including a shared encoder layer providing computational operations for image processing tasks and branches which span over encoder layers providing at least partly different computational operations for the image processing tasks. Each branch is associated with a certain image processing task. Receptive encoder layer field sizes and assessment measures are calculated, each assessment measure referring to a combination of a certain encoder architecture and a certain image processing task, and including information regarding matching quality of characteristic scale distribution associated with the assessment measure to the receptive field sizes of the encoder layers. The assessment measures are compared and a comparison result established. An encoder architecture is selected based on the comparison result. And wherein Ivan Calculates the receptive field size investigating the size of an image portion of an input image which has an influence on the features of the feature map and therefore the output of a certain encoder layer. The receptive field sizes of the encoder layers may, for example, be calculated based on the convolutional properties, for example kernel size and stride factor).
Claim 11 is similar in scope to claim 2 therefore the claims are rejected under similar rationale.
Claim 16 is similar in scope to claim 7 therefore the claims are rejected under similar rationale.
Claims 5-6 and 14-15 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Tran et al. US Patent Application Publication US 20200143227 A1 (hereinafter Tran) in view of Zehngut et al. US Patent Application Publication US 20220147680 A1 (hereinafter Zehngut) and further in view of Wen et al. US Patent Application Publication US 20230409867 A1 (hereinafter Wen) further in view of Wu et al. US Patent Application US 11748615 A1 (hereinafter Wu).
Regarding claim 5, Tran, Zehngut and Wen do not teach wherein the first score corresponds to a sum of a first group of scores corresponding to the first group of neural network layers.
However in analogous of selecting an artificial intelligence model in neural architecture search, Wu teaches wherein the first score corresponds to a sum of a first group of scores corresponding to the first group of neural network layers (¶ 21, wherein Wu describes search process of DNAS engine 32 is extremely fast compared with previous reinforcement learning (RL) based methods. In some examples, the loss used to train stochastic super net 34 consists of both the cross-entropy loss that leads to better accuracy and the latency loss that penalizes the network's latency on a target device. To estimate the latency of an architecture, the latency of each operator in the search space may be measured and provided by user 29 for storage within latency LUT 36 from which DNAS engine 32 computes the overall latency of a candidate convolutional neural network model by summing the latency of each operator. Using this model, the latency of architectures can quickly be estimated by an otherwise enormous search space. Further, the techniques make the latency differentiable with respect to layer-wise block choices, which can be leveraged by DNAS engine 32).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Wu with Tran, Zehngut and Wen by incorporating the method of wherein the first score corresponds to a sum of a first group of scores corresponding to the first group of neural network layers of Wu into the method for selecting an artificial intelligence (AI) model in neural architecture search (NAS) of Tran, Zehngut and Wen for the purpose of incorporating techniques that make the latency differentiable with respect to layer-wise block choices, which can be leveraged by DNAS engine. (Wu: ¶ 21).
Regarding claim 6, Tran as modified by Zehngut, Wen and Wu teach wherein the second score corresponds to a sum of a second group of scores corresponding to the second group of neural network layers (¶ 21, wherein Wu describes search process of DNAS engine 32 is extremely fast compared with previous reinforcement learning (RL) based methods. In some examples, the loss used to train stochastic super net 34 consists of both the cross-entropy loss that leads to better accuracy and the latency loss that penalizes the network's latency on a target device. To estimate the latency of an architecture, the latency of each operator in the search space may be measured and provided by user 29 for storage within latency LUT 36 from which DNAS engine 32 computes the overall latency of a candidate convolutional neural network model by summing the latency of each operator. Using this model, the latency of architectures can quickly be estimated by an otherwise enormous search space. Further, the techniques make the latency differentiable with respect to layer-wise block choices, which can be leveraged by DNAS engine 32).
Claim 14 is similar in scope to claim 5 therefore the claims are rejected under similar rationale.
Claim 15 is similar in scope to claim 6 therefore the claims are rejected under similar rationale.
Claims 8 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Tran et al. US Patent Application Publication US 20200143227 A1 (hereinafter Tran) in view of Zehngut et al. US Patent Application Publication US 20220147680 A1 (hereinafter Zehngut) and further in view of Wen et al. US Patent Application Publication US 20230409867 A1 (hereinafter Wen) further in view of Ozcan et al. US Patent Application Publication US 20220122313 A1 (hereinafter Ozcan).
However in analogous of selecting an artificial intelligence model in neural architecture search, Wu teaches wherein each of the plurality of candidate AI models is a zero-cost proxy mode (FIGS. 6A-6B, [0021] wherein Ozcan describes as illustrated in FIGS. 6A-6B describes the stability test of Recurrent-MZ inference. FIG. 6A shows an additive Gaussian noise with zero mean and a standard variance of σ was injected into each DPM to test the stability of Recurrent-MZ inference. The output images and difference maps (with respect to ground truth) with no injected noise (σ=0) and σ=1 μm and σ=0.5 μm are shown. FIG. 6B shows plots of the NRMSE-σ boxplot (z=4.6 μm—left; z=6.8 μm—right). NRMSE values were calculated over 50 random tests. The difference maps were normalized by the maximum difference between the input images and the ground truth).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Ozcan with Tran, Zehngut and Wen by incorporating the method of wherein each of the plurality of candidate AI models is a zero-cost proxy mode of Ozcan into the method for selecting an artificial intelligence (AI) model in neural architecture search (NAS) of Tran, Zehngut and Wen for the purpose of incorporating a network that is based on a convolutional recurrent network design, which combines the advantages of both convolutional neural networks and recurrent neural networks in processing sequential inputs (Ozcan: [0078]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HASSAN MRABI whose telephone number is (571)272-8875. The examiner can normally be reached on Monday-Friday, 7:30am-5pm. Alt, Friday, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HASSAN MRABI/Examiner, Art Unit 2144
Prosecution Insights