DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Liu et al. (US 2022/0198272) and Li et al. (US 2019/0051290).
Regarding claim 1, Liu et al. discloses a method comprising:
receiving, at a device and via a user interface, a selection of a labeled training dataset and a selection of a target training dataset, wherein the target training dataset is captured from a target domain (“In some embodiments, the standard training datasets 132 may or may not be labeled or partially labeled, but the domain-specific training datasets 134 may be properly labeled” at paragraph 0030, second to last sentence);
forming, by the device, a domain-adapted training dataset (“The domain-specific training datasets 134 may be collected respectively from one or more specific domains” at paragraph 0032, line 4);
training, by the device, a machine learning model using the domain-adapted training dataset (“In some embodiments, the backbone neural network 210 may be used as a foundation to train detectors 220 for specific domains. Here, a domain may refer to a geographic location, a spatial and/or temporal unit, or another suitable form. As a simple example, a domain may be represented by a surveillance camera monitoring a spatial unit (e.g., an isle in a store, a park)” at paragraph 0037, line 1); and
pruning, by the device, the machine learning model to form a domain-adapted model for the target domain (“In some embodiments, the neural network pruning component 126 in the back-end processing component 122 may be configured to reduce the number of active neurons at least in the first neural network. In some embodiments, the first neural network within the second neural network is trained based on a large amount of data and includes a large number of neurons, while each branch extended from the first neural network is trained based on a small amount of data and thus include a small number of neurons. For this reason, an effective way to reduce the size of the second neural network may be pruning the first neural network therein. In some embodiments, the pruning process may also sparsify the neurons in the branches. In some embodiments, the pruning and training of the second neural network may be executed as an iterative process” at paragraph 0033, line 1).
Liu et al. does not explicitly disclose that the target training dataset is an unlabeled training dataset.
However, Liu et al. mentions above that labeling of the target dataset “may” occur, meaning it is not mandatory. Additionally, Liu et al. demonstrates that each image is represented by a specific camera, as indicated by a camera identifier. Therefore, by forming a set from a specific camera identifier would preclude the need for labeling.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize such a dataset for the target dataset to cut down on the time cost associated with such a task.
Liu et al. further does not explicitly disclose that the domain-adapted training dataset is formed by pruning the labeled training dataset based on the unlabeled training dataset.
Li et al. teaches a method in the same field of endeavor of neural network domain adaptation, comprising:
forming, by the device, a domain-adapted training dataset by pruning the standard training dataset based on the target training dataset (“Method 300 optionally proceeds to OPERATION 340, where the training data, including the source domain data 130 and the associated target domain data 140, are pruned according the domain characteristics of the target domain” at paragraph 0054, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the labeled training dataset of Liu et al. by pruning as taught by Li et al. to create a curated training dataset that most closely matches the expected target domain (see Li et al. at paragraph 0054).
Regarding claim 11, Liu et al. discloses an apparatus, comprising:
a network interface to communicate with a computer network (“The computer system 600 may include a network interface 618 coupled to bus 602. Network interface 618 may provide a two-way data communication coupling to one or more network links that are connected to one or more local networks” at paragraph 0067, line 1);
a processor coupled to the network interface and configured to execute one or more processes (“one or more hardware processor(s) 604 coupled with bus 602 for processing information” at paragraph 0063, line 3); and
a memory configured to store a process that is executed by the processor (“Main memory 606 also may be used for storing temporary variables or other intermediate information during execution of instructions executable by processor(s) 604” at paragraph 0064, line 5), the process when executed configured to:
receive, via a user interface, a selection of a labeled training dataset and a selection of a target training dataset, wherein the target training dataset is captured from a target domain (“In some embodiments, the standard training datasets 132 may or may not be labeled or partially labeled, but the domain-specific training datasets 134 may be properly labeled” at paragraph 0030, second to last sentence);
form a domain-adapted training dataset (“The domain-specific training datasets 134 may be collected respectively from one or more specific domains” at paragraph 0032, line 4);
train a machine learning model using the domain-adapted training dataset (“In some embodiments, the backbone neural network 210 may be used as a foundation to train detectors 220 for specific domains. Here, a domain may refer to a geographic location, a spatial and/or temporal unit, or another suitable form. As a simple example, a domain may be represented by a surveillance camera monitoring a spatial unit (e.g., an isle in a store, a park)” at paragraph 0037, line 1); and
prune the machine learning model to form a domain-adapted model for the target domain (“In some embodiments, the neural network pruning component 126 in the back-end processing component 122 may be configured to reduce the number of active neurons at least in the first neural network. In some embodiments, the first neural network within the second neural network is trained based on a large amount of data and includes a large number of neurons, while each branch extended from the first neural network is trained based on a small amount of data and thus include a small number of neurons. For this reason, an effective way to reduce the size of the second neural network may be pruning the first neural network therein. In some embodiments, the pruning process may also sparsify the neurons in the branches. In some embodiments, the pruning and training of the second neural network may be executed as an iterative process” at paragraph 0033, line 1).
Liu et al. does not explicitly disclose that the target training dataset is an unlabeled training dataset.
However, Liu et al. mentions above that labeling of the target dataset “may” occur, meaning it is not mandatory. Additionally, Liu et al. demonstrates that an image is represented by a specific camera, as indicated by a camera identifier. Therefore, by forming a set from a specific camera identifier would preclude the need for labeling.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize such a dataset for the target dataset to cut down on the time cost associated with such a task.
Liu et al. further does not explicitly disclose that the domain-adapted training dataset is formed by pruning the labeled training dataset based on the unlabeled training dataset.
Li et al. teaches an apparatus in the same field of endeavor of neural network domain adaptation, comprising:
forming a domain-adapted training dataset by pruning the standard training dataset based on the target training dataset (“Method 300 optionally proceeds to OPERATION 340, where the training data, including the source domain data 130 and the associated target domain data 140, are pruned according the domain characteristics of the target domain” at paragraph 0054, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the labeled training dataset of Liu et al. by pruning as taught by Li et al. to create a curated training dataset that most closely matches the expected target domain (see Li et al. at paragraph 0054).
Regarding claim 20, Liu et al. discloses a tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process (“Main memory 606 also may be used for storing temporary variables or other intermediate information during execution of instructions executable by processor(s) 604” at paragraph 0064, line 5) comprising:
receiving, at a device and via a user interface, a selection of a labeled training dataset and a selection of a target training dataset, wherein the target training dataset is captured from a target domain (“In some embodiments, the standard training datasets 132 may or may not be labeled or partially labeled, but the domain-specific training datasets 134 may be properly labeled” at paragraph 0030, second to last sentence);
forming, by the device, a domain-adapted training dataset (“The domain-specific training datasets 134 may be collected respectively from one or more specific domains” at paragraph 0032, line 4);
training, by the device, a machine learning model using the domain-adapted training dataset (“In some embodiments, the backbone neural network 210 may be used as a foundation to train detectors 220 for specific domains. Here, a domain may refer to a geographic location, a spatial and/or temporal unit, or another suitable form. As a simple example, a domain may be represented by a surveillance camera monitoring a spatial unit (e.g., an isle in a store, a park)” at paragraph 0037, line 1); and
pruning, by the device, the machine learning model to form a domain-adapted model for the target domain (“In some embodiments, the neural network pruning component 126 in the back-end processing component 122 may be configured to reduce the number of active neurons at least in the first neural network. In some embodiments, the first neural network within the second neural network is trained based on a large amount of data and includes a large number of neurons, while each branch extended from the first neural network is trained based on a small amount of data and thus include a small number of neurons. For this reason, an effective way to reduce the size of the second neural network may be pruning the first neural network therein. In some embodiments, the pruning process may also sparsify the neurons in the branches. In some embodiments, the pruning and training of the second neural network may be executed as an iterative process” at paragraph 0033, line 1).
Liu et al. does not explicitly disclose that the target training dataset is an unlabeled training dataset.
However, Liu et al. mentions above that labeling of the target dataset “may” occur, meaning it is not mandatory. Additionally, Liu et al. demonstrates that an image is represented by a specific camera, as indicated by a camera identifier. Therefore, by forming a set from a specific camera identifier would preclude the need for labeling.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize such a dataset for the target dataset to cut down on the time cost associated with such a task.
Liu et al. further does not explicitly disclose that the domain-adapted training dataset is formed by pruning the labeled training dataset based on the unlabeled training dataset.
Li et al. teaches a method in the same field of endeavor of neural network domain adaptation, comprising:
forming, by the device, a domain-adapted training dataset by pruning the standard training dataset based on the target training dataset (“Method 300 optionally proceeds to OPERATION 340, where the training data, including the source domain data 130 and the associated target domain data 140, are pruned according the domain characteristics of the target domain” at paragraph 0054, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the labeled training dataset of Liu et al. by pruning as taught by Li et al. to create a curated training dataset that most closely matches the expected target domain (see Li et al. at paragraph 0054).
Regarding claims 2 and 12, Liu et al. discloses a method and apparatus wherein the labeled training dataset comprises video data labeled with classification labels indicative of at least one of: types of objects depicted in the video data or events depicted in the video data (“The specific application of the neural network is not limited in this application, which may include object detection or identification, object counting, classification, prediction, recommendation (e.g., in a recommender system), segmentation, voice recognition, various signal processing, or any other suitable applications” at paragraph 0034, line 4; therefore for specific object detection, the images are labeled with information related to the object detection and identification).
Regarding claims 5 and 15, Liu et al. discloses a method and apparatus wherein the process when executed is further configured to:
receive, via the user interface, layer-wise pruning ratios, wherein the apparatus prunes the machine learning model according to the layer-wise pruning ratios (“The embodiment illustrated in route 440 may include the following steps: pruning the first neural network until reaching a sparsity ratio; training the second neural network based on the pruned first neural network and the one or more second training datasets until an objective function converges; increasing the sparsity ratio; and further training the pruned second neural network based on the one or more second training datasets until the objective function converges; and further pruning the further trained second neural network to reach the increased sparsity ratio until an exit condition is met” at paragraph 0048).
Regarding claims 7 and 17, Liu et al. discloses a method and apparatus wherein the labeled training dataset comprises data captured at least in part at a location that differs from that of the target domain (“For example, if a surveillance camera monitoring a domain is used to represent the domain, the camera's identifier may be used as the domain identifier. The camera's identifier may include its IP address, MAC address, serial number, or another suitable identifier” at paragraph 0040, line 5; as the large datasets in Figure 2 comprise a variety of locations and conditions that differ from the specific domains as demonstrated in the small datasets).
Regarding claims 9 and 19, Liu et al. discloses a method and apparatus wherein the process when executed is further configured to:
deploy the domain-adapted model to an edge device in the target domain (the discussion of paragraph 0027 is to demonstrate that the invention as disclosed is able to be deployed to edge devices by creating a domain specific model for the particular use case).
Claim(s) 3, 4, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Liu et al. and Li et al. as applied to claims 1 and 11 above, and further in view of Castelli et al. (US 6,298,351).
Regarding claims 3 and 13, the Liu et al. and Li et al. combination discloses the elements of claims 1 and 11 above.
The Liu et al. and Li et al. combination does not explicitly disclose that the device uses a parametric approach to prune the labeled training dataset based on the unlabeled training dataset.
Castelli et al. teaches a method and apparatus in the same field of endeavor of training set construction for classification, wherein the device uses a parametric approach to prune the labeled training dataset based on the unlabeled training dataset (“The output of the supervised training, the set of rules, is then input to a supervised classification technique 812, which instantiates the classifier. The results of the instantiation are compared against one or more predetermined conditions to determine if the modification is suitable, INQUIRY 814. For example, in one embodiment, a stopping condition is that the classification error does not reduce anymore when the training set is revised” at col. 6, line 63; “One example of a supervised classification technique 808 (FIG. 8) is progressive classification. Progressive classification is a methodology for constructing robust classifiers that relies on the properties of transforms used for source coding. Any kind of existing classifier, both parametric (Maximum Likelihood, Bayesian) and non-parametric (Nearest Neighbor, Learning Vector Quantization, Neural Networks, CART, to mention just a few), can be implemented as a progressive classifier” at col. 7, line 41).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a classification as taught by Castelli et al. to prune the training set of the Liu et al. and Li et al. combination to allow for “determining at least one datum of the training set is incorrect and reconstructing the at least one datum to provide a modified training set” (Castelli et al. at col. 1, line 61).
Regarding claims 4 and 14, the Liu et al. and Li et al. combination discloses the elements of claims 1 and 11 above.
The Liu et al. and Li et al. combination does not explicitly disclose that the device uses a non-parametric approach to prune the labeled training dataset based on the unlabeled training dataset.
Castelli et al. teaches a method and apparatus in the same field of endeavor of training set construction for classification, wherein the device uses a parametric approach to prune the labeled training dataset based on the unlabeled training dataset (“The output of the supervised training, the set of rules, is then input to a supervised classification technique 812, which instantiates the classifier. The results of the instantiation are compared against one or more predetermined conditions to determine if the modification is suitable, INQUIRY 814. For example, in one embodiment, a stopping condition is that the classification error does not reduce anymore when the training set is revised” at col. 6, line 63; “One example of a supervised classification technique 808 (FIG. 8) is progressive classification. Progressive classification is a methodology for constructing robust classifiers that relies on the properties of transforms used for source coding. Any kind of existing classifier, both parametric (Maximum Likelihood, Bayesian) and non-parametric (Nearest Neighbor, Learning Vector Quantization, Neural Networks, CART, to mention just a few), can be implemented as a progressive classifier” at col. 7, line 41; “During unsupervised classification, the training set (and/or the validation set) is clustered by any one of various clustering schemes, such as k-means, self-organization map, or vector quantization” at col. 4, line 62).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a classification as taught by Castelli et al. to prune the training set of the Liu et al. and Li et al. combination to allow for “determining at least one datum of the training set is incorrect and reconstructing the at least one datum to provide a modified training set” (Castelli et al. at col. 1, line 61).
Claim(s) 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Liu et al. Li et al. as applied to claims 1 and 11 above, and further in view of Tripp (US 2016/0182558).
The Liu et al. and Li et al. combination discloses the elements of claims 1 and 11 above.
The Liu et al. and Li et al. combination does not explicitly disclose that the process when executed is further configured to: provide, to the user interface, samples of the domain-adapted training dataset for display.
Tripp teaches a method and apparatus in the same field of endeavor of machine learning wherein the process when executed is further configured to:
provide, to the user interface, samples of the training dataset for display (“At step 504, the individual findings comprising the subset X of S are then presented to the user. This training set is provided in any convenient manner, such as via a visual display, printed output, or the like” at paragraph 0084, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to present the training data of the Liu et al. and Li et al. combination for display as taught by Tripp to allow the user to verify the relevance of the constructed training set.
Claim(s) 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Liu et al. and Li et al. as applied to claims 1 and 11 above, and further in view of Saruta et al. (US 2019/0303746).
The Liu et al. and Li et al. combination discloses the elements of claims 1 and 11 above.
The Liu et al. and Li et al. combination does not explicitly disclose that the process when executed is further configured to: provide, to the user interface, a comparison of an accuracy of the domain-adapted model to that of the machine learning model.
Saruta et al. teaches a method and apparatus in the same field of endeavor of adaptive domain machine learning, wherein the process when executed is further configured to:
provide, to the user interface, a comparison of an accuracy of the domain-adapted model to that of the machine learning model (“FIG. 17 is a view exemplarily showing a GUI for accepting selection of an NN. More specifically, FIG. 17 shows the way the display unit 508 displays neural networks A, B, and C having undergone adaptive domain learning” at paragraph 0118, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to display the accuracy data as taught by Saruta et al. in the system of the Liu et al. and Li et al. combination to allow the user to evaluate the performance of the domain adaptation (see Saruta et al. at paragraphs 0118 and 0119).
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Liu et al. and Li et al. as applied to claim 9 above, and further in view of Murez et al. (US 2019/0244107).
The Liu et al. and Li et al. combination discloses the elements of claims 1 and 11 above.
The Liu et al. and Li et al. combination does not explicitly disclose that the domain-adapted model assesses sensor data captured in the target domain.
Murez et al. teaches a method in the same field of endeavor of domain adaption for machine learning, wherein the domain-adapted model assesses sensor data captured in the target domain (“FIG. 8 depicts an image processing system (element 800) comprising a convolutional neural network (CNN). CNNs are made of two parts: a deep feature extractor module (element 802), which maps an input image obtained from a target domain sensor (element 804) into a feature space, and a linear classifier (or regressor) module (element 806), which maps the features to the desired output, such as a labeled target domain (element 808)” at paragraph 0057, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize target domain sensor data as taught by Murez et al. in the domain adaption of the Liu et al. and Li et al. combination as it “boosts the transferability of knowledge (e.g., classification, segmentation) from one-domain to another” (Murez et al. at paragraph 0057, last sentence).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 12:00-8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672