DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the claims filed 10/23/2023.
Claims 1-20 are presented for examination.
Specification
The disclosure is objected to because of the following informalities:
[122] recites “adjust parameters values” and “adjusts parameters values”, and FIG. 6, box 606 recites “Adjust parameters values” of the machine learning model; in each instance “parameters values” should read “parameter values” as well as change “adjusts” to “adjust”.
Appropriate correction is required.
Claim Objections
Claim 20 is objected to because of the following informalities: in the body of the claim, “cause the one or more computers to perform operation comprising” should read “cause the one or more computers to perform operations comprising”, consistent with the corresponding language of claims 1 and 18.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent claims 1, 18, and 20 each recite the limitation training the machine learning model using one or more final data augmentation policies generated from the compact search space to perform a three-dimensional perception task. There is insufficient antecedent basis for the limitation (the machine learning model) in each of these claims. No machine learning model is recited earlier in claim 1, claim 18, or claim 20; the preceding limitations recite only data augmentation policies, search spaces, and hyperparameters. Because the model that is to be trained is introduced for the first time by the definite article the, a person of ordinary skill in the art cannot determine with reasonable certainty whether a machine learning model is a positively recited element of the claim or what model is intended. See MPEP 2173.05(e). For purposes of examination, and consistent with the specification at [014] and [022] which describe training a machine learning model to perform a particular machine learning task using one or more data augmentation policies, the limitation is interpreted as training a machine learning model.
Claims 2-17 depend from claim 1 and claim 19 depends from claim 18; they incorporate and do not cure this defect, and are rejected for the same reason.
Claim 8 recites the limitations performing a grid search ... to determine the optimal value based on a performance attained by the proxy machine learning model as a result of the training. There is insufficient antecedent basis for the limitations (the proxy machine learning model) and (the training) in this claim. Claim 8 depends from claim 6, and neither a proxy machine learning model nor any training of such a model is recited in claim 8 or in any of its ancestor claims (claim 6, claim 4, claim 3, or claim 1). A proxy machine learning model and its training are first introduced only in claim 7, which is a separate claim that also depends from claim 6 and is therefore not an ancestor of claim 8. Antecedent basis is evaluated along the dependency chain, and a sibling claim cannot supply antecedent basis. See MPEP 2173.05(e). As a result, the metric on which the optimal value is based and the entity that performs the grid search are left uncertain. For purposes of examination, and consistent with the specification at [103]-[104] which describe selecting values, training a proxy machine learning model for a predetermined number of training iterations, and then performing a grid search based on the performance attained by that proxy model, claim 8 is interpreted as introducing a proxy machine learning model as a result of training the proxy machine learning model.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9-14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. "Improving 3D Object Detection through Progressive Population Based Augmentation" (2020) (hereinafter Cheng) in view of Cubuk et al. "RandAugment: Practical automated data augmentation with a reduced search space" (2020) (hereinafter Cubuk).
Cheng and Cubuk were disclosed in an IDS dated 12/2/2024.
Regarding independent claim 1, Cheng teaches a method (Abstract, "we present the first attempt to automate the design of data augmentation policies for 3D object detection"):
obtaining data defining an original search space of a plurality of candidate data augmentation policies (Section 3.1, pages 3-4, "In the proposed search space, an augmentation policy consists of N augmentation operations"; where the proposed search space (an original search space) is populated by augmentation policies each serving as a candidate over which the search ranges);
wherein each candidate data augmentation policy defines a procedure for processing a training input comprising three-dimensional point cloud data to generate a transformed training input (Section 3.1, page 4 Fig.1 "An augmentation policy is defined by a list of distinct augmentation operations and the corresponding augmentation parameters"; where each augmentation policy (a candidate data augmentation policy) is a list of operations applied to point clouds during training to produce augmented point clouds);
and wherein each candidate data augmentation policy has one or more respective local hyperparameters corresponding to different aspects of the procedure defined by the candidate data augmentation policy (Section 3.1, page 4, "there are 8 augmentation operations and 29 operation parameters in the proposed search space"; where each operation of the policy carries its own parameters governing how that operation is applied, which are the per-aspect parameters (local hyperparameters) of the procedure);
and training the machine learning model (interpreted as a machine learning model per the 35 U.S.C. 112(b) rejection set forth above) using one or more final data augmentation policies to perform a three-dimensional perception task (Abstract, page 1, "PPBA continues to effectively improve the StarNet and PointPillars detectors…without an incurred cost at inference time"; teaches training the model (training a machine learning model) with the searched policies to perform three-dimensional object detection at inference time (using one or more final data augmentation policies to perform a three-dimensional perception task)).
Cheng does not expressly teach generating, from the original search space, a compact search space that has one or more global hyperparameters; and using augmentation policies generated from the compact search space.
However, Cubuk teaches generating, from the original search space, a compact search space that has one or more global hyperparameters (Section 1, page 2, "We hence introduce a vastly simplified search space for data augmentation containing 2 interpretable hyper-parameters"; where the large per-operation augmentation parameter space (the original search space) is replaced by a much smaller space (a compact search space) defined by two interpretable parameters; Section 3, page 4, "a single global distortion M may suffice for parameterizing all transformations"; where a single distortion value is shared across every transformation (one or more global hyperparameter)); and using augmentation policies generated from the compact search space (Section 1, page 2, "We hence introduce a vastly simplified search space for data augmentation containing 2 interpretable hyper-parameters").
Because Cheng and Cubuk are analogous art and within the same field of endeavor, specifically automated search for data augmentation policies used to train machine learning perception models, they address the same problem solving area of reducing the cost of searching a large data augmentation policy search space, accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the reduced-search-space reparameterization of Cubuk with the per-operation policy search framework of Cheng, with a reasonable expectation of success, such that the original per-operation search space is reparameterized into a smaller set of hyperparameters shared across operations, to teach generating, from the original search space, a compact search space that has one or more global hyperparameters; and using data augmentation policies generated from the compact search space. This modification would have been motivated by the desire to remove the separate, expensive search phase and allow a simple grid search to find a good policy while reducing the search cost of the large point-cloud policy search space (Cubuk Section 1, page 2)
Regarding dependent claim 2, Cheng, in view of Cubuk, teach the method of claim 1, wherein each local hyperparameter (see Cheng Appendix Table 8, page 18, where each operation parameter (a local hyperparameter) is assigned a range of values that can be searched, "The range of augmentation parameters that can be searched by Progressive Population Based Augmentation algorithm for each operation") or global hyperparameter (see Cubuk Section 3, page 4, where the shared distortion magnitude and the count of transformations (global hyperparameters) each range over an integer scale) has a respective space of possible values (see Cheng Appendix Table 8, page 18, where each operation parameter is assigned its own value range, for example a probability parameter assigned the range zero to one (a respective space of possible values); see also Cubuk Section 3, page 4, where each transformation resides on an integer scale from zero to ten, "each transformation resides on an integer scale from 0 to 10").
Regarding dependent claim 3, Cheng, in view of Cubuk, teach the method of claim 1, wherein generating the compact search space that has one or more global hyperparameters comprises, for each candidate data augmentation policy in the plurality of candidate data augmentation policies: defining each of the one or more respective local hyperparameters of the candidate data augmentation policy in terms of one or more of the global hyperparameters by using one or more normalization coefficients, one or more normalization constants, or both (see Cubuk Section 3, page 4, where each transformation's per-operation magnitude is expressed on the same linear scale parameterized by the single shared magnitude, so that the per-operation parameter is defined in terms of the global parameter through a linear mapping, "we employ the same linear scale for indicating the strength of each transformation", "a single global distortion M may suffice for parameterizing all transformations").
Regarding dependent claim 4, Cheng, in view of Cubuk, teach the method of claim 3, wherein defining each of the one or more respective local hyperparameters of the candidate data augmentation policy in terms of one or more of the global hyperparameters comprises, for each of the one or more respective local hyperparameters: determining a value of a normalization coefficient, a value of a normalization constant, or both (see Cubuk Section 3, page 4, where each transformation resides on the same linear scale and a per-transformation scale factor fixes how the shared distortion value maps onto that transformation's own strength, so that the per-transformation scale factor (a value of a normalization coefficient) is set for each transformation, "we employ the same linear scale for indicating the strength of each transformation"); and computing a function that has a first term representing a product between a global hyperparameter and the normalization coefficient having the determined value (see Cubuk Section 3, page 4, where the per-operation magnitude is obtained from the single shared distortion magnitude (a global hyperparameter) on the common linear scale, that is, as the shared magnitude multiplied by the per-transformation scale factor (the product term), "each transformation resides on an integer scale from 0 to 10 where a value of 10 indicates the maximum scale for a given transformation"), a second term representing the normalization constant having the determined value, or both the first term and the second term (see Cubuk Section 3, page 4, where the linear scale carries the transformation's own range so that the mapped value includes a per-transformation range endpoint added to the scaled magnitude (a second term), "each transformation resides on an integer scale from 0 to 10 where a value of 10 indicates the maximum scale for a given transformation"; the claim recites the first term, the second term, or both in the alternative, and the product term computed from the shared magnitude on the common linear scale satisfies the recited function under the alternative).
Regarding dependent claim 5, Cheng, in view of Cubuk, teach the method of claim 4, wherein determining the value of the normalization coefficient, the value of the normalization constant, or both comprises: determining, for each respective local hyperparameter of the candidate data augmentation policy, the value of the normalization coefficient, the value of the normalization constant, or both that are different from the value of the normalization coefficient, the value of the normalization constant, or both of another respective local hyperparameter of the candidate data augmentation policy (see Cubuk Section 3, page 4, where each transformation has its own scale mapping the shared magnitude onto that transformation's own value range, so that the per-transformation factor and offset differ from one transformation to another, "each transformation resides on an integer scale from 0 to 10 where a value of 10 indicates the maximum scale for a given transformation").
Regarding dependent claim 6, Cheng, in view of Cubuk, teach the method of claim 4, wherein determining the value of the normalization coefficient, the value of the normalization constant, or both comprises: determining an optimal value of each of the one or more respective local hyperparameters of the candidate data augmentation policy (see Cheng Section 4.1, page 8, where trial models are trained and evaluated by the mean average precision metric to discover the best per-operation parameters, "16 trials are trained to optimize the mAP") ; determining the value of the normalization coefficient, the value of the normalization constant, or both based on the optimal values of the respective local hyperparameters and the respective space of possible values of each of the one or more global hyperparameters (see Cubuk Section 3, page 4, where the shared-magnitude scale is set so that the global magnitude over its range reproduces the per-operation magnitudes, "we observe that the learned magnitude for each transformation follows a similar schedule during training", "a single global distortion M may suffice for parameterizing all transformations").
Regarding dependent claim 7, Cheng, in view of Cubuk, teach the method of claim 6, wherein determining the optimal value of each of the one or more respective local hyperparameters of the candidate data augmentation policy comprises, for each candidate data augmentation policy in the plurality of candidate data augmentation policies: selecting a value from the respective space of possible values of each of the one or more respective local hyperparameters (see Cheng Section 3.2 and Algorithm 1, page 4-5, where augmentation operation parameters are sampled from the search space for each trial, "all model parameters and augmentation parameters are randomly initialized") ; and training a proxy machine learning model for a predetermined number of training iterations using the candidate data augmentation policy in accordance with the one or more respective local hyperparameters that have the selected values (see Cheng Section 4.1, page 8, where each trial model (a proxy machine learning model) is trained for a set number of steps with its sampled parameters before being evaluated, "We train the first iteration for 3,000 steps, and all subsequent iterations for 1,000 steps with batch size 64").
Regarding dependent claim 9, Cheng, in view of Cubuk, teach the method of claim 1, wherein the three-dimensional perception task comprises a three-dimensional object detection task (see Cheng Abstract, where the trained model performs three-dimensional object detection over point clouds, "Data augmentation has been widely adopted for object detection in 3D point clouds").
Regarding dependent claim 10, Cheng, in view of Cubuk, teach the method of claim 9, wherein obtaining data defining the original search space of the plurality of candidate data augmentation policies comprises: obtaining data defining a point cloud augmentation policy which defines a procedure of modifying a point cloud generated by using a LIDAR sensor in a training input (see Cheng Section 1, page 1, Appendix Table 7, page 18, where the FrustumDropout operation operates on the LIDAR point cloud (a point cloud generated by using a LIDAR sensor) and drops points to produce a modified point cloud, "All points are first converted to spherical coordinates, and then a point is randomly selected") while accounting for occlusion from a point of view of the LIDAR sensor (see Cheng Section 1, page 1, Appendix Table 7, page 18, where the points are converted to spherical coordinates from the sensor origin and only the points within a frustum around a selected point, defined by phi and theta angle widths and a distance threshold, are dropped, so that the dropping follows the angular view from the sensor (accounting for occlusion from a point of view of the LIDAR sensor), "All points in the frustum around that point within a given phi, theta angle width and distance to the original greater than a given value are dropped randomly").
Regarding dependent claim 11, Cheng, in view of Cubuk, teach the method of claim 9, wherein the procedure for processing the training input defined by each candidate data augmentation policy comprise one of: dropping out data points, replicating data points, changing background data points, rotating data points, scaling data points, adding noisy data points, translating data points, or flipping data points within the training input (see Cheng Section 3.1, page 4, Appendix Table 7, page 18, where the search space includes operations that drop out points, rotate points, scale points, translate points, flip points, and add noise to points; the recited group is an alternative grouping for which the prior art need teach only one member, and Cheng_PPBA teaches several, "GroundTruthAugmentor, RandomFlip, WorldScaling, GlobalTranslateNoise, FrustumDropout, FrustumNoise, RandomRotation and RandomDropLaserPoints").
Regarding dependent claim 12, Cheng, in view of Cubuk, teach the method of claim 11, wherein for the candidate data augmentation policy that defines the procedure of dropping out data points, the candidate data augmentation policy has a first local hyperparameter corresponding to a probability of applying the procedure to the training input (see Cheng Appendix Table 8, page 18, where the RandomDropout operation carries a dropout probability parameter governing whether and how often a point is dropped (a first local hyperparameter corresponding to a probability of applying the procedure), "dropout probability") and a second local hyperparameter corresponding to a ratio of dropped out data points to all data points in the training input (see Cheng Appendix Table 8, page 18, where the FrustumDropout operation carries a separate parameter for the probability of dropping a point that sets the proportion of points removed from the cloud (a second local hyperparameter corresponding to a ratio of dropped out data points to all data points), "the probability of dropping a point").
Regarding dependent claim 13, Cheng, in view of Cubuk, teach the method of claim 12, wherein defining the first local hyperparameter of the candidate data augmentation policy in terms of one or more of the global hyperparameters comprises: computing a function that has a single term representing a product between a first global hyperparameter and a normalization coefficient having a first determined value (see Cubuk Section 3, page 4, where every per-operation strength value is placed on one common linear scale parameterized by the single shared global magnitude, so that the per-operation value is obtained as the shared global magnitude multiplied by a per-transformation scale factor, that is, a single product term in the global parameter, "we employ the same linear scale for indicating the strength of each transformation", "each transformation resides on an integer scale from 0 to 10 where a value of 10 indicates the maximum scale for a given transformation").
Regarding dependent claim 14, Cheng, in view of Cubuk, teach the method of claim 12, wherein defining the second local hyperparameter of the candidate data augmentation policy in terms of one or more of the global hyperparameters comprises: computing a function that has (i) a first term representing a product between a second global hyperparameter and a normalization coefficient having a second determined value (see Cubuk Section 3, page 4, where the shared distortion magnitude (a second global hyperparameter) is placed on the common linear scale by a per-transformation scale factor (a normalization coefficient), so that the mapped value includes the shared magnitude multiplied by the per-transformation scale factor (a first term representing a product), "we employ the same linear scale for indicating the strength of each transformation") and (ii) a second term representing a normalization constant having a third determined value (see Cubuk Section 3, page 4, where each transformation has its own value range with its own range endpoint, so that the linear-scale mapping adds the per-transformation range endpoint to the scaled magnitude (a second term representing an additive per-transformation range endpoint), "each transformation resides on an integer scale from 0 to 10 where a value of 10 indicates the maximum scale for a given transformation").
Regarding dependent claim 16, Cheng, in view of Cubuk, teach the method of claim 1, wherein training the machine learning model using the one or more final data augmentation policies generated from the compact search space comprises: generating each final data augmentation policy based on selecting a respective final value from the respective space of possible values of each of the one or more global hyperparameters (see Cubuk Section 3 and Figure 3, page 4, where a final policy is fixed by choosing a value of the count of transformations and a value of the shared distortion magnitude from their respective integer ranges (selecting a respective final value from the respective space of possible values of each global hyperparameter), "The resulting algorithm contains two parameters N and M"), each final data augmentation policy comprising some or all of the plurality of candidate data augmentation policies in the original search space (see Cubuk Section 3, page 4, where the chosen count of transformations selects which of the available transformations make up the policy, so that each final policy comprises some or all of the available transformations (some or all of the plurality of candidate data augmentation policies)), each candidate data augmentation policy included in the final data augmentation policy having respective final values of the one or more respective local hyperparameters that are defined by the respective final values of the one or more global hyperparameters (see Cubuk Section 3, page 4, where the per-operation magnitude of each selected transformation is then fixed by the chosen shared distortion magnitude on the common linear scale, so that the per-operation values follow from the global value, "a single global distortion M may suffice for parameterizing all transformations").
Regarding dependent claim 17, Cheng, in view of Cubuk, teach the method of claim 1, wherein training the machine learning model using the one or more final data augmentation policies generated from the compact search space comprises: selecting a batch of training data from a training data set (see Cheng Section 4.1, page 8, where the detector is trained on successive batches of the training data drawn from the training set at batch size 64 (selecting a batch of training data from a training data set), "with batch size 64"); generating an augmented batch of training data by transforming the training inputs in the batch of training data in accordance with the one or more final data augmentation policies (see Cheng Section 3.2, page 4-5, where each augmentation operation of the policy is applied to the training inputs as the training progresses, producing augmented inputs for the batch (generating an augmented batch of training data by transforming the training inputs in accordance with the policies), "each operation is applied stochastically and its parameters evolve as the training progresses"); and adjusting parameters values of the machine learning model based on the augmented batch of training data (see Cheng Section 3.2, page 4-5, where the model parameters are updated by stochastic gradient descent on the objective evaluated over the augmented training data (adjusting parameters values of the machine learning model based on the augmented batch), "the model θ is optimizing").
Regarding claims 18-19, these system claims that are substantially the same as the method of claims 1 and 17, respectively. Thus, claims 18-19 are rejected for the same reasons as claims 1 and 17. In addition, Cheng teaches a system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations (Section 4, pages 8-10, where the search and training are carried out on computers executing the described procedure on stored data and parameters).
Regarding claim 20, this is a non-transitory computer storage medium claim that is substantially the same as the method of claim 1. Thus, claim 20 is rejected for the same reason as claim 1. In addition, Cheng teaches a non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations (Section 4, page 8-10, where the described search and training procedure is carried out by computers executing stored instructions).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of Cubuk, as applied in the rejection of claim 9, and further view of Yin et al. "Center-based 3D Object Detection and Tracking" (2021) (hereinafter Yin).
Yin was disclosed in an IDS dated 12/2/2024.
Regarding dependent claim 15, Cheng, in view of Cubuk, teach all the elements of claim 9.
Cheng and Cubuk do not expressly teach wherein the machine learning model is a neural network comprising one or more multi-layer perceptions, a fully convolutional backbone, a heatmap prediction head, and a bounding box regression head.
However, Yin teaches wherein the machine learning model is a neural network comprising one or more multi-layer perceptions (Section 4.1 Two-Stage CenterPoint, page 11787, where the point features are processed by a multi-layer perceptron (one or more multi-layer perceptions), "we concatenate the extracted point-features and pass them through an MLP", and the second-stage network is itself a multi-layer perceptron, Section 4.2 Architecture, page 11788, "Our second-stage uses a shared two-layer MLP"), a fully convolutional backbone (Section 3, page 11786, where the detector architecture uses a standard fully convolutional backbone, "The CenterNet architecture uses a standard fully convolutional image backbone"), a heatmap prediction head (Section 4 CenterPoint, page 11786, where a center heatmap head produces a heatmap peak at the center of each detected object (a heatmap prediction head), "Center heatmap head. The center-head's goal is to produce a heatmap peak at the center location of any detected object"), and a bounding box regression head (Section 1 Introduction, page 11785, where the model regresses from the object center to the three-dimensional bounding box size and orientation (a bounding box regression head), "it regresses to all other object properties such as 3D size, orientation, and velocity")).
Because Cheng, in view of Cubuk, and Yin are analogous art and within the same field of endeavor, specifically machine learning perception over three-dimensional point clouds, and Yin is reasonably pertinent to the problem of selecting a detector architecture for the three-dimensional object detection task trained with the augmented data, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to implement the three-dimensional object detection model of the Cheng and Cubuk combination with the center-based detector of Yin, with a reasonable expectation of success, to teach wherein the machine learning model is a neural network comprising one or more multi-layer perceptions, a fully convolutional backbone, a heatmap prediction head, and a bounding box regression head. This modification would have been motivated by the desire to use a detector that reduces the search space of the detector and outperforms anchor-based detectors on point cloud benchmarks (Yin Section 1, page 2).
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims as well as overcoming the 35 U.S.C. 112(b) rejections. None of the applied references teaches or suggests the claim 8 limitations. At best Cheng discloses: Data augmentation has been widely adopted for object detection in 3D point clouds. However, all previous related efforts have focused on manually designing specific data augmentation methods for individual architectures. In this work, we present the first attempt to automate the design of data augmentation policies for 3D object detection. We introduce the Progressive Population Based Augmentation (PPBA) algorithm, which learns to optimize augmentation strategies by narrowing down the search space and adopting the best parameters discovered in previous iterations. On the KITTI 3D detection test set, PPBA improves the StarNet detector by substantial margins on the moderate difficulty category of cars, pedestrians, and cyclists, outperforming all current state-of-the-art single-stage detection models and Cubuk discloses: Recent work has shown that data augmentation has the potential to significantly improve the generalization of deep learning models. Recently, automated augmentation strategies have led to state-of-the-art results in image classification and object detection. While these strategies were optimized for improving validation accuracy, they also led to state-of-the-art results in semi-supervised learning and improved robustness to common corruptions of images. An obstacle to a large-scale adoption of these methods is a separate search phase which increases the training complexity and may substantially increase the computational cost. Additionally, due to the separate search phase, these approaches are unable to adjust the regularization strength based on model or dataset size. Automated augmentation policies are often found by training small models on small datasets and subsequently applied to train larger models. In this work, we remove both of these obstacles. RandAugment has a significantly reduced search space which allows it to be trained on the target task with no need for a separate proxy task. Furthermore, due to the parameterization, the regularization strength may be tailored to different model and dataset sizes. Accordingly, claim 8 contains allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
MEYERSON et al., US 2019/0130257 A1, (May 2, 2019) (ABSTRACT The technology disclosed identifies parallel ordering of shared layers as a common assumption underlying existing deep multitask learning (MTL) approaches. This assumption restricts the kinds of shared structure that can be learned between tasks. The technology disclosed demonstrates how direct approaches to removing this assumption can ease the integration of information across plentiful and diverse tasks. The technology disclosed introduces soft ordering as a method for learning how to apply layers in different ways at different depths for different tasks, while simultaneously learning the layers themselves. Soft ordering outperforms parallel ordering methods as well as single-task learning across a suite of domains. Results show that deep MTL can be improved while generating a compact set of multipurpose functional primitives, thus aligning more closely with our understanding of complex real-world processes).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KUANG FU CHEN whose telephone number is (571)272-1393. The examiner can normally be reached M-F 9:00-5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KC CHEN/Primary Patent Examiner, Art Unit 2143
Prosecution Insights