DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Claim(s) 1-20 do not use “means for” (or “step for”) language, or generic placeholders for "means” coupled with functional language without recitation of sufficient structure for carrying out the claimed functions and therefore do not invoke 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 2, 6, 11-14, 16 and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Riquelme et al. (“Scaling Vision with Sparse Mixture of Experts” – Cited on IDS).[claim 1]
Regarding claim 1, Riquelme discloses a system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to implement a neural network that is configured to process a network input and to generate a network output for the network input, the neural network comprising a sequence of one or more network blocks (Figure 1, Section I and Abstract; computer vision system comprising a plurality of MLPs stored on a separate device), the sequence comprising one or more expert network blocks configured to perform operations comprising:
obtaining a block input that represents an intermediate representation of the network input, the block input comprising a plurality of elements (Page 3, Section 2.2, input images are processed as a sequence of patches (plurality of elements));
determining a plurality of sub-inputs from the block input, each sub-input comprising a respective different subset of the plurality of elements of the block input (Page 3, Section 3.2; input image is divided into a grid of equal-sized patches);
for each of a plurality of expert subnetworks of the expert network block:
processing the plurality of sub-inputs to generate a respective score for each sub- input (Page 7, Section 4.1; compute a priority score s(x) on each token);
selecting one or more of the sub-inputs according to the respective scores (Page 7, Section 4.1; sort tokens based on their maximum routing weight to favor “most important” tokens); and
for each selected sub-input, processing the selected sub-input using the expert subnetwork to generate a respective sub-output (Page 7, Section 4.1-4.2; assigning tokens to experts according to sort; see also Pages 25-26 and Section C.1);
for each of the plurality of sub-inputs, processing the sub-outputs corresponding to the sub-input generated by respective expert subnetworks to generate a combined sub-output for the sub-input (Figure 1, Sections 4-5; processing sub-inputs using expert subnetworks to generated sub-outputs); and
generating a block output by combining the respective combined sub-outputs for the plurality of sub-inputs (Figure 1, summing sub-outputs from the subnetwork experts according to weights so form block outputs).[claim 2]
Regarding claim 2, Riquelme discloses wherein each expert subnetwork is configured to process a same number of sub-inputs (Figure 1).[claim 6]
Regarding claim 6, Riquelme discloses wherein each sub-input is processed by at most a threshold number b different expert subnetworks (Figure 1; Page 7, Section 4.1-4.2; Pages 25-26, Section C.1). Note that the claim does not define the particular threshold number, thus the threshold number may be chosen to be higher than the number of different expert subnetworks which process each sub-input.[claim 11]
Regarding claim 11, Riquelme discloses wherein, for each of the plurality of sub-inputs, processing the sub-outputs corresponding to the sub-input generated by respective expert subnetworks to generate a combined sub-output for the sub-input comprises:
determining a weighted sum of the sub-outputs corresponding to the sub-input, wherein each sub-output is weighted by the score of the sub-input for the expert subnetwork that generated the sub-output (Figure 1; weighted sum of sub-outputs according to assigned weights).[claim 12]
Regarding claim 12, Riquelme discloses wherein the operations of at least some of the plurality of expert subnetworks are executed in parallel (Figure 1, Device 1 through Device E in parallel arrangement).[claim 13]
Regarding claim 13, Riquelme discloses wherein the operations of at least some of the plurality of expert subnetworks are executed by respective different devices (Figure 1, Device 1 through Device E).[claim 14, 16 and 20]
Claims 14, 16 and 20 are method claims corresponding to apparatus claims 1, 2 and 6. Therefore, claims 14, 16 and 20 are analyzed and rejected as previously discussed with respect to claims 1, 2 and 6.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Riquelme et al. (“Scaling Vision with Sparse Mixture of Experts” – Cited on IDS).[claim 15]
Regarding claim 15, see the rejection of claims 1 and 14 above. However, Riquelme does not explicitly teach implementing the claimed system/method using one or more non-transitory computer storage media storing instructions which are executed by one or more computers to cause the one or more computers to implement the neural network as claimed.
Official Notice is taken that it is well known in the art to implement neural network systems/methods such as the computer vision system/method taught by Riquelme using one or more non-transitory computer storage media storing instructions which are executed by one or more computers to cause the one or more computers to implement the neural network. By using a one or more non-transitory computer storage media storing instructions which are executed by one or more computers to cause the one or more computers to implement the neural network, the neural network may be implemented using commercially available off-the-shelf programmable hardware. Additionally, the neural network may be updated by rewriting the stored instructions, thereby allowing for updates without the need to change the underlying hardware of the system.
Therefore, it would have been obvious to implement the system/method taught by Riquelme using one or more non-transitory computer storage media storing instructions which are executed by one or more computers to cause the one or more computers to implement the neural network so that the system/method may be implemented using commercially available off-the-shelf programmable hardware and updated without the need to change the underlying hardware of the system.
Allowable Subject Matter
Claims 3-5, 7-10 and 17-19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.[claims 3-5, 7-10 and 17-19]
Regarding claims 3-5, 7-10 and 17-19, the prior art does not teach or reasonably suggest the particular requirements of these claims. While the prior art teaches similar systems/methods (see e.g. Riquelme as discussed above), the prior art does not teach the specific relationships recited in the claims.
The prior art additionally teaches other systems/methods for routing inputs in mixture-of-experts neural network systems, e.g. El-Yaniv et al. (US 12,572,794 B2 – Figure 5), Chen et al. (US 2022/0383200 A1 – Figure 1), Odena et al. (US 2019/0236438 A1 – Figure 1) and Shazeer et al. (US 2019/0251423 A1 – Figure 1), but like Riquelme, these references do not teach or reasonably suggest the specific relationships recited in the claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY J HENN whose telephone number is (571)272-7310. The examiner can normally be reached Monday-Friday ~10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Timothy J Henn/Primary Examiner, Art Unit 2639
Prosecution Insights