CTNF 18/347,877 CTNF 88277 DETAILED ACTION 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claims 1-11 and 16-24 are presented for examination. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA 1. Claim s 1-3, 7, 10, 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Publication No. US2022/0114453) in view of Zhou et al. (US Publication No. US 2022/0304602) . Regarding claim 1, Lee et al. meets the claim limitations, as follows: A method comprising: obtaining an embedding neural network , wherein the embedding neural network is pretrained to embed inputs (para[0117], Fig. 5, i.e. acquire information about a pretrained artificial neural network step 501) performing a first progressive pruning stage, wherein the first progressive pruning stage includes a first pruning of the embedding neural network and a first fine-tuning of the embedding neural network (para[0126]-[0128], Fig. 5, i.e. perform a pruning operation on the artificial neural network based on the pruning-evaluation operation variable step 504 and update pruning-evaluation operation and average inference task accuracy step 505 and 507 for fine tuning the neural network by converting the corresponding weight value.) ; performing a second progressive pruning stage based on an output of the first progressive pruning stage, wherein the second progressive pruning stage includes a second pruning of the embedding neural network and a second fine-tuning of the embedding neural network (para[0129]-[0136], Fig. 5, i.e. determine to additionally perform the pruning-evaluation operation based on a preset epoch and the task accuracy of the repruned artificial neural network. As shown in steps 504-509, the second pruning operation is performed when the updated learning weight λ.sub.s is greater than the lower limit threshold λ.sub.min of the learning weight in operation 506, and update pruning-evaluation operation and average inference task accuracy step 505 and 507 for fine tuning the neural network.) . Lee et al. teaches that input various types of signals (for example, any one of a voice recognition signal, an object recognition signal, a video recognition signal, and a biological information recognition signal) (para[0110]), but Lee et al. does not explicitly disclose the following claim limitations: wherein the embedding neural network is pretrained to embed inputs from a plurality of modalities into a multimodal embedding space; However, in the same field of endeavor Zhou et al. discloses the deficient claim limitations, as follows: wherein the embedding neural network is pretrained to embed inputs from a plurality of modalities into a multimodal embedding space (Fig. 6 para[0049], i.e. multimodal input 601 is used to pretrain AI model 602); Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention was made to modify the teachings of Lee with Zhou to use AI model pretrained on multimodal inputs, the motivation being to use additional data inputs to increase system accuracy (para[0051]). Regarding claim 2, the rejection of claim 1 is incorporated herein. Lee et al. meets the claim limitations, as follows: The method of claim 1, further comprising: determining a number of progressive pruning procedures; and iteratively performing the number of progressive pruning procedures on the embedding neural network (para[0019], i.e. The processor may be configured to determine whether to additionally perform the pruning-evaluation operation based on a preset epoch and the task accuracy of the repruned neural network.). Regarding claim 3, the rejection of claim 1 is incorporated herein. Lee et al. meets the claim limitations, as follows: The method of claim 1, further comprising: computing a contrastive learning loss, wherein the first fine-tuning is based on the contrastive learning loss (para[0100], i.e. the controller 320 may determine a loss function based on a variable used to determine a threshold of a weight that is used as a standard for performing, the determined pruning weight, and the task accuracy of the pruned artificial neural network and may update a threshold determination variable to decrease the loss function.) . Regarding claim 7, the rejection of claim 1 is incorporated herein. Lee et al. meets the claim limitations, as follows: The method of claim 1, further comprising: identifying a subset of layers of the embedding neural network for fine-tuning, wherein the first pruning is performed on the subset of layers of the embedding neural network (Fig. 5, para[0126]-[0128], i.e. In operation 504, when a weight value of the pretrained artificial neural network or the pruned artificial neural network in the previous pruning-evaluation operation is less than a reference threshold of pruning, the apparatus may convert the corresponding weight value to 0 or may decrease the weight value to a smaller value.). Regarding claim 10, the rejection of claim 1 is incorporated herein. Lee et al. meets the claim limitations, as follows: The method of claim 1, further comprising: pretraining the embedding neural network prior to the first progressive pruning stage (Fig. 5, i.e. a pretrained artificial neural network.). Regarding claim 21, all claimed limitations are set forth and rejected as per discussion for claim 1. Here, a Non-transitory computer readable medium is equivalent to the memory (Lee i.e. Memory 440 in Fig. 4) Regarding claim 22, all claimed limitations are set forth and rejected as per discussion for claim 2. Regarding claim 23, all claimed limitations are set forth and rejected as per discussion for claim 3 . 07-21-aia AIA 2. Claim s 4-5, 8-9 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Publication No. US2022/0114453) in view of Zhou et al. (US Publication No. US 2022/0304602) and further in view of Li et al. (US Publication No. US 2023/0084203) . Regarding claim 4, the rejection of claim 1 is incorporated herein. Lee et al. and Zhou fails to teach the claim limitations, as follows: The method of claim 1, further comprising: adding a temporary indicator layer to the embedding neural network; and training the embedding neural network together with the temporary indicator layer prior to the first progressive pruning stage, wherein the first pruning is based on the temporary indicator layer. However, in the same field of endeavor Li et al. discloses the deficient claim limitations, as follows: adding a temporary indicator layer to the embedding neural network (para[0034], i.e. an indicator mask) ; and training the embedding neural network together with the temporary indicator layer prior to the first progressive pruning stage, wherein the first pruning is based on the temporary indicator layer (Fig. 2, [0050], i.e. A pruned network (e.g., a pruned CNN candidate 240) with the new architecture and weights may then be obtained based on the channel indicator masks 235 and the CNN weights 225 (the pruned CNN candidate 240 has different layer attributes than the original CNN architecture); Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention was made to modify the teachings of Lee and Zhou with Li to use an indicator mask for pruning neural network, the motivation being to increase network operation performance (para[0002]). Regarding claim 5, the rejection of claim 4 is incorporated herein. Li et al. meets the claim limitations, as follows: The method of claim 4, further comprising: determining a pruning threshold for the first pruning (para[0140], i.e. setting the threshold value based on the a confidence threshold for keeping a channel or not); identifying an element of the temporary indicator layer that is less than the pruning threshold (Fig. 2 para[0050], i.e. obtain channel indicator masks 235 by a pruning threshold (1/0 stands for keeping/pruning a channel, respectively) ; and pruning a neuron of the embedding neural network corresponding to the element of the temporary indicator layer (Fig. 2 para[0050], i.e. A pruned network (e.g., a pruned CNN candidate 240) with the new architecture and weights may then be obtained based on the channel indicator masks 235 and the CNN weights 225 (the pruned CNN candidate 240 has different layer attributes than the original CNN architecture,) . Regarding claim 8, the rejection of claim 7 is incorporated herein. Lee et al. and Zhou fails to teach the claim limitations, as follows: The method of claim 7, wherein: the subset of layers includes a feed-forward layer. However, in the same field of endeavor Li et al. discloses the deficient claim limitations, as follows: the subset of layers includes a feed-forward layer (Fig. 4A para [0106], i.e. determining weights for the pruned neural network by applying a feed-forward neural network to a function of the updated embeddings and corresponding importance vectors for the pruned neural network.); Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention was made to modify the teachings of Lee and Zhou with Li to use a feed-forward neural network for pruning neural network, the motivation being to increase network operation performance (para[0002]). Regarding claim 9, the rejection of claim 1 is incorporated herein. Lee et al. and Zhou fails to teach the claim limitations, as follows: The method of claim 1, further comprising: adding an adapter layer to the embedding neural network prior to the first progressive pruning stage. However, in the same field of endeavor Li et al. discloses the deficient claim limitations, as follows: adding an adapter layer to the embedding neural network prior to the first progressive pruning stage (para[0034], i.e. an indicator mask). Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention was made to modify the teachings of Lee and Zhou with Li to use an indicator mask for pruning neural network, the motivation being to increase network operation performance (para[0002]). Regarding claim 24, all claimed limitations are set forth and rejected as per discussion for claim 4 . 07-21-aia AIA 3. Claim s 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Publication No. US2022/0114453) in view of Zheng et al. (US Publication No. US 2022/0335043) . Regarding claim 16, Lee et al. meets the claim limitations, as follows: An apparatus comprising: at least one processor (Fig. 7, para[0154], i.e. a controller 720 (for example, one or more processors) ; the apparatus further comprising a memory including instructions executable by the at least one processor (Fig. 7, para[0154], i.e. a memory 710) ; and an embedding neural network comprising parameters stored in the memory and, wherein the embedding neural network is trained using a progressive pruning procedure (para[0126]-[0128], Fig. 5, i.e. perform a pruning operation on the artificial neural network based on the pruning-evaluation operation variable step 504 and update pruning-evaluation operation and average inference task accuracy step 505 and 507 for fine tuning the neural network by converting the corresponding weight value.) . Lee et al. does not explicitly disclose the following claim limitations: An embedding neural network trained to convert a query into a multimodal embedding space to obtain a query embedding; However, in the same field of endeavor Zheng et al. discloses the deficient claim limitations, as follows: An embedding neural network trained to convert a query into a multimodal embedding space to obtain a query embedding (Fig. 2A, para[0021],[0042],[0035] i.e. The language encoding module 230 may receive and convert the search query 210 into intermediate query encoding 240 which may consist of one or more embeddings.); Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention was made to modify the teachings of Lee with Zheng to convert the search query into intermediate query encoding, the motivation being to yield more user-friendly applications and improved accuracy in command recommendations (para[0022]). Regarding claim 17, Lee et al. meets the claim limitations, as follows: The apparatus of claim 16, further comprising: a training component configured to perform the progressive pruning procedure by performing a first progressive pruning stage, wherein the first progressive pruning stage includes a first pruning of the embedding neural network and a first fine-tuning of the embedding neural network (para[0126]-[0128], Fig. 5, i.e. perform a pruning operation on the artificial neural network based on the pruning-evaluation operation variable step 504 and update pruning-evaluation operation and average inference task accuracy step 505 and 507 for fine tuning the neural network by converting the corresponding weight value.) ; performing a second progressive pruning stage based on an output of the first progressive pruning stage, wherein the second progressive pruning stage includes a second pruning of the embedding neural network and a second fine-tuning of the embedding neural network (para[0129]-[0136], Fig. 5, i.e. determine to additionally perform the pruning-evaluation operation based on a preset epoch and the task accuracy of the repruned artificial neural network. As shown in steps 504-509, the second pruning operation is performed when the updated learning weight λ.sub.s is greater than the lower limit threshold λ.sub.min of the learning weight in operation 506, and update pruning-evaluation operation and average inference task accuracy step 505 and 507 for fine tuning the neural network.) . Regarding claim 18, Zheng et al. meets the claim limitations, as follows: The apparatus of claim 16, further comprising: a user interface configured to receive the query and display a search result obtained in response to the query (Fig. 1 and 4, para[0039] i.e. the client device 120 may also include a local command recommendation model 124 for providing local command recommendation services. ). Regarding claim 19, Zheng et al. meets the claim limitations, as follows: The apparatus of claim 16, wherein: the embedding neural network comprises a transformer architecture (para[0021], i.e. transformer-based deep learning mode). Regarding claim 20, Zheng et al. meets the claim limitations, as follows: The apparatus of claim 16, further comprising: a search engine configured to compare the query embedding to a candidate embedding (Fig. 2A i.e. command prediction module) . Allowable Subject Matter 12-151-08 AIA 07-43 12-51-08 1. Claim s 6 and 11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATE H LUO whose telephone number is (571)270-5635. The examiner can normally be reached on 8:00-5:00PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, can be reached on Alejandro Rivero (571)270-3641. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /KATE H LUO/Primary Examiner, Art Unit 6216 Application/Control Number: 18/347,877 Page 2 Art Unit: 6216 Application/Control Number: 18/347,877 Page 3 Art Unit: 6216 Application/Control Number: 18/347,877 Page 4 Art Unit: 6216 Application/Control Number: 18/347,877 Page 5 Art Unit: 6216 Application/Control Number: 18/347,877 Page 6 Art Unit: 6216 Application/Control Number: 18/347,877 Page 7 Art Unit: 6216 Application/Control Number: 18/347,877 Page 8 Art Unit: 6216 Application/Control Number: 18/347,877 Page 9 Art Unit: 6216 Application/Control Number: 18/347,877 Page 10 Art Unit: 6216 Application/Control Number: 18/347,877 Page 11 Art Unit: 6216 Application/Control Number: 18/347,877 Page 12 Art Unit: 6216 Application/Control Number: 18/347,877 Page 13 Art Unit: 6216 Application/Control Number: 18/347,877 Page 14 Art Unit: 6216
Prosecution Insights