Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending for examination. Claims 1 and 13 are independent.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
According to the first part of the analysis, in the instant case, claims 1-12 are directed to a method, and claims 13-20 are directed to an apparatus. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Regarding Claim 1:
2A Prong 1:
restoring an input map corresponding to a second layer of the second neural network based on a delta input map corresponding to the second layer and the base input map; (This step for restoring an input map (e.g. adding input maps (see claim 5) is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
obtaining a delta output map corresponding to the second layer based on a difference between the base output map and the output map corresponding to the second layer; (This step for obtaining a delta output map based on a difference between output maps is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
A method performed by at least one processor based on a first neural network and a second neural network, the method comprising: (This step for performing with processor and neural networks is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).)
obtaining a base output map corresponding to a first layer of the first neural network by applying, to the first layer, a base input map corresponding to the first layer; (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural network as a tool to perform the abstract idea - see MPEP 2106.05(f).)
obtaining an output map corresponding to the second layer by applying, to the second layer, the restored input map corresponding to the second layer; (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural network as a tool to perform the abstract idea - see MPEP 2106.05(f).)
storing the base output map and the delta output map. (This step directed to storing information, is understood to be insignificant extra- solution activity and data gathering. See MPEP 2106.05(g).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are insignificant extra solution activity in combination of generic computer functions that are implemented to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
A method performed by at least one processor based on a first neural network and a second neural network, the method comprising: (This step for performing with processor and neural networks is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).)
obtaining a base output map corresponding to a first layer of the first neural network by applying, to the first layer, a base input map corresponding to the first layer; (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural network as a tool to perform the abstract idea - see MPEP 2106.05(f).)
obtaining an output map corresponding to the second layer by applying, to the second layer, the restored input map corresponding to the second layer; (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural network as a tool to perform the abstract idea - see MPEP 2106.05(f).)
storing the base output map and the delta output map. (This step is directed to storing information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity as identified by the court (MPEP 2106.05(d)(ll)(IV)))))
The additional elements as disclosed above in combination of the abstract idea
are not sufficient to amount to significantly more than the judicial exception as they are
well, understood, routine and conventional activity as disclosed in combination
of generic computer functions that are implemented to perform the disclosed abstract idea above.
Regarding Claim 13: see the rejection of claim 1 above. Same rationale applies.
2A Prong 2 & 2B: The claim recites another additional element “An apparatus including a first neural network and a second neural network, the apparatus comprising: at least one processor; and at least one memory including instructions executable by the processor to:” (mere instructions to apply the exception using generic computer components - see MPEP 2106.05(f))
Regarding Claims 2 and 14
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
wherein the storing the base output map and the delta output map comprises: storing the base output map as a base input map corresponding to a subsequent layer of the first layer; and storing the delta output map as a delta input map corresponding to a subsequent layer of the second layer. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the base output map and the delta output map - See MPEP 2106.05(h).)
Regarding Claim 3
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
wherein the first neural network is obtained by fine-tuning a pretrained base model based on transfer learning for a first task, and the second neural network is obtained by fine-tuning the base model based on transfer learning for a second task. (Training a neural network is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 4
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
wherein the first layer corresponds to a layer of a base model, and the second layer corresponds to the same layer of the base model as the first layer. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the layers - See MPEP 2106.05(h).)
Regarding Claims 5 and 15
2A Prong 1:
wherein the restoring the input map corresponding to the second layer comprises: adding the base input map and the delta input map to restore the input map corresponding to the second layer. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claims 6 and 16
2A Prong 1:
compressing the output map corresponding to the first layer to obtain the base output map corresponding to the first layer. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B:
wherein the obtaining the base output map corresponding to the first layer comprises: obtaining an output map corresponding to the first layer by applying the base input map to the first layer (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural networks as a tool to perform the abstract idea - see MPEP 2106.05(f).); and
Regarding Claims 7 and 17
2A Prong 1:
wherein the obtaining the delta output map corresponding to the second layer comprises: compressing the output map corresponding to the second layer (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).); and subtracting, from the base output map, the compressed output map corresponding to the second layer to obtain the delta output map corresponding to the second layer. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claims 8 and 18
2A Prong 1:
wherein the storing the base output map and the delta output map comprises: compressing the delta output map based on a characteristic of a sparse matrix of the delta output map (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).); and encoding the compressed delta output map and the base output map to store the base output map and the delta output map. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claims 9 and 19
2A Prong 1:
restoring the first layer based on the base weight and the first delta weight; (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).) and
restoring the second layer based on the base weight and the second delta weight. (This step is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2:
further comprising: obtaining a base weight corresponding to the first layer and the second layer based on a base model corresponding to the first neural network and the second neural network; (This step is directed receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
obtaining a first delta weight corresponding to the first layer and a second delta weight corresponding to the second layer; (This step is directed receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
2B:
further comprising: obtaining a base weight corresponding to the first layer and the second layer based on a base model corresponding to the first neural network and the second neural network; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
obtaining a first delta weight corresponding to the first layer and a second delta weight corresponding to the second layer; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
Regarding Claims 10 and 20
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
further comprising: storing, as a base input map corresponding to a next layer of the first neural network, a first map obtained by applying input data to an initial layer of the first neural network; (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the base input map- See MPEP 2106.05(h).) and
storing, as a delta input map corresponding to a next layer of the second neural network, a difference between the first map and a second map obtained by applying the input data to an initial layer of the second neural network. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the delta input map - See MPEP 2106.05(h).)
Regarding Claim 11
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
wherein the first neural network and the second neural network comprise a sequence of a plurality of layers performing a series of operations on input data (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or merely applying a generic neural networks as a tool to perform the abstract idea - see MPEP 2106.05(f).), and the first neural network and the second neural network are different in at least a portion of the layers. (The specification of data to be stored is understood to be a field of use limitation. The limitation further specifies the first and second neural network- See MPEP 2106.05(h).)
Regarding Claim 12
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1. (This step is adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 9-15, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 20200134506 A1, hereinafter "Wang") in view of Krishnan et al. (US 20230401831 A1, hereinafter " Krishnan").
Regarding Claim 1
Wang discloses: A method performed by at least one processor ([Para 0086-0089 and Fig 5-6] disclose a processor.) based on a first neural network and a second neural network ([Para 0004, 0037, 0041, and Fig 2] describes a teacher model (i.e. first neural network) and student model (i.e. second neural network).), the method comprising:
obtaining a base output map corresponding to a first layer of the first neural network by applying, to the first layer, a base input map corresponding to the first layer ([Para 0008, 0037, 0043-0045, 0059, and Fig 2] describes obtaining an output of a teacher model (i.e. first neural network) by applying an input x (i.e. base input map). [para 0037] Examiner interprets the teacher model being a neural network having a first layer.);
restoring an input map corresponding to a second layer of the second neural network based on a delta input map corresponding to the second layer and the base input map ([Para 0035, 0037, 0058-0060 0072-0073 and Fig 2-4] describes adding a variation Δ (i.e. delta input map) to the input x (i.e. base input map) of the student model (i.e. second neural network). [para 0037] Examiner interprets the layer of the student model as a second layer.);
obtaining an output map corresponding to the second layer by applying, to the second layer, the restored input map corresponding to the second layer ([Para 0035-0036, 0041, 0058-0060, and Fig 2-4] describes obtaining an output of a student model (i.e. second neural network) by applying input “(xi+Δ)” (i.e. restored input map).);
obtaining a delta output map corresponding to the second layer based on a difference between the base output map and the output map corresponding to the second layer ([Para 0032, 0035, 0041-0048, 0060, 0074 and Fig 2-4] describes calculating a difference between output of a teacher model (i.e. base output map) and output of a student model (i.e. output map corresponding to the second layer) to train the student model. Examiner interprets the difference as a delta output map.); and
Wang does not explicitly disclose: storing the base output map and the delta output map.
However, Krishnan discloses in the same field of endeavor: storing the base output map and the delta output map. ([Para 0022, 0029 0040, and Fig 1-4] describes storing outputs from a teacher model (i.e. base output map) and outputs from a student model (i.e. delta output map).)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Storing outputs disclosed by Krishnan into the method of Model Training disclosed by Wang to store model outputs. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Storing outputs disclosed by Krishnan as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to store and track model outputs.
Regarding Claim 13
Wang in view of Krishnan discloses: An apparatus including a first neural network and a second neural network ([Para 0004, 0037, 0041, and Fig 2], Wang describes a teacher model (i.e. first neural network) and student model (i.e. second neural network).), the apparatus comprising: at least one processor; and at least one memory including instructions executable by the processor to ([Para 0086-0089 and Fig 5-6], Wang disclose the apparatus.): (Claim 13 is an apparatus claim that corresponds to claim 1 and the rest of the limitations are rejected on the same ground)
Regarding Claim 2
Wang in view of Krishnan discloses: The method of claim 1, wherein the storing the base output map and the delta output map comprises: storing the base output map as a base input map corresponding to a subsequent layer of the first layer; and storing the delta output map as a delta input map corresponding to a subsequent layer of the second layer. ([Para 0037, 0041, 0058-0079] Wang describes iterative training. [Para 0022, 0029 0040, and Fig 1-4], Krishnan describes storing outputs from a teacher classification model and outputs from a student classification model.)
Regarding Claim 3
Wang in view of Krishnan discloses: The method of claim 1, wherein the first neural network is obtained by fine-tuning a pretrained base model based on transfer learning for a first task ([Para 0004-0006, 0037-0041, 0043-0046, Fig 3-4], Wang describes training the teacher model and pre-training. [Para 0060 and 0075] Wang describes different tasks.), and the second neural network is obtained by fine-tuning the base model based on transfer learning for a second task. ([Para 0004-0006, 0032, 0035-0041, 0043-0046, Fig 3-4], Wang describes training the student model. [Para 0060 and 0075] Wang describes different tasks.)
Regarding Claim 4
Wang in view of Krishnan discloses: The method of claim 1, wherein the first layer corresponds to a layer of a base model, and the second layer corresponds to the same layer of the base model as the first layer. ([Para 0035, 0037, 0058-0060 0072-0073 and Fig 2-4] describes the teacher student models corresponding to the same input (e.g. input layer).)
Regarding Claim 5
Wang in view of Krishnan discloses: The method of claim 1, wherein the restoring the input map corresponding to the second layer comprises: adding the base input map and the delta input map to restore the input map corresponding to the second layer. ([Para 0035, 0037, 0058-0060 0072-0073 and Fig 2-4], Wang describes adding a variation Δ (i.e. delta input map) to the input x (i.e. base input map) of the student model (i.e. second neural network).
Regarding Claim 9
Wang in view of Krishnan discloses: The method of claim 1, further comprising: obtaining a base weight corresponding to the first layer and the second layer based on a base model corresponding to the first neural network and the second neural network; obtaining a first delta weight corresponding to the first layer and a second delta weight corresponding to the second layer; restoring the first layer based on the base weight and the first delta weight; and restoring the second layer based on the base weight and the second delta weight. ([Para 0036- 0041, 0109], Wang describes processing weights across layers.)
Regarding Claim 10
Wang in view of Krishnan discloses: The method of claim 1, further comprising: storing, as a base input map corresponding to a next layer of the first neural network, a first map obtained by applying input data to an initial layer of the first neural network; and storing, as a delta input map corresponding to a next layer of the second neural network, a difference between the first map and a second map obtained by applying the input data to an initial layer of the second neural network. ([Para 0028, 0030 and Fig 1] Krishnan describes storing outputs from a teacher classification model (i.e. base output map) and outputs from a student classification model (i.e. delta output map).)
Regarding Claim 11
Wang in view of Krishnan discloses: The method of claim 1, wherein the first neural network and the second neural network comprise a sequence of a plurality of layers performing a series of operations on input data, and the first neural network and the second neural network are different in at least a portion of the layers. ([Para 0004, 0037, 0041, and Fig 2] Wang describes a teacher model (i.e. first neural network) and student model (i.e. second neural network).)
Regarding Claim 12
Wang in view of Krishnan discloses: A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1. ([Para 0080 and Fig 8-9], Krishnan discloses non-transitory memory and processor. [Para 0086-0089 and Fig 5-6], Wang also disclose a processor.)
Regarding Claim 14
(Claim 14 recites analogous limitations to claim 2 and therefore is rejected on the same ground as claim 2.)
Regarding Claim 15
(Claim 15 recites analogous limitations to claim 5 and therefore is rejected on the same ground as claim 5.)
Regarding Claim 19
(Claim 19 recites analogous limitations to claim 9 and therefore is rejected on the same ground as claim 9.)
Regarding Claim 20
(Claim 20 recites analogous limitations to claim 10 and therefore is rejected on the same ground as claim 10.)
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Krishnan and Sridhar et al. (US 20210279595, hereinafter "Sridhar").
Regarding Claim 6
Wang in view of Krishnan discloses: The method of claim 1, wherein the obtaining the base output map corresponding to the first layer comprises: obtaining an output map corresponding to the first layer by applying the base input map to the first layer; ([Para 0008, 0037, 0043-0045, 0059, and Fig 2] describes obtaining an output of a teacher model (i.e. first neural network) by applying an input x (i.e. base input map).)
Wang in view of Krishnan does not explicitly disclose: compressing the output map corresponding to the first layer to obtain the base output map corresponding to the first layer.
However, Sridhar discloses in the same field of endeavor: compressing the output map corresponding to the first layer to obtain the base output map corresponding to the first layer. ([Para 0038 0058-0061 0089-0096 and Fig 3-5] describes a compression block for a teacher model that generates compressed data (i.e. compressing the output).)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Compression disclosed by Sridhar into the method of Wang in view of Krishnan to compress an output map. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Compression disclosed by Sridhar as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to generate a less computationally-intensive model having fewer learned parameters.
Regarding Claim 16
(Claim 16 recites analogous limitations to claim 6 and therefore is rejected on the same ground as claim 6.)
Allowable Subject Matter
Claims 7-8 and 17-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
None of these references taken alone or in combination with the prior art of record disclose the same steps for compressing the output map corresponding to the second layer and the delta output map, in combination with the remaining elements and features of the claimed invention. It is for these reasons that the applicants’ invention defines over the prior art of record.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. KO et al. (US 20240119571 A1) describes a first and second neural network with deblurring. Habibian et al. (US 20230154169) describes a delta input.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TEWODROS E MENGISTU whose telephone number is (571)270-7714. The examiner can normally be reached Mon-Fri 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ABDULLAH KAWSAR can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TEWODROS E MENGISTU/Examiner, Art Unit 2127
Prosecution Insights