DETAILED ACTION
Remarks
This office action is issued in response to communication filed on 5/16/2022. Claims 1-20 are pending in this Office Action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The abstract of the disclosure is objected to because it includes a file name in the abstract “51172519.docx”. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Objections
Claim objected to because of the following informalities: Claims 7-8 and 11 recite “the respective transformed input vectors” which lacks proper antecedent basis because claim 1 only recite “respective transformed input “vector” in singular form Appropriate correction is required.
Allowable Subject Matter
Claims 3-6 and 8-10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
2. Claims 1 - 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1:
Step 1: Statutory Category ?: Yes. claim 1 recites a system (i.e., a “machine”) which is statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitations: “for each position, applying a first set of transformations to the respective input vector at the position to generate a respective transformed input vector at the position, each respective transformed input vector having a second number of channels; generating a respective spatially transformed input vector at each of the positions, comprising applying a feedforward spatial transformation that integrates information across the plurality of positions; and generating an output sequence for the block by, for each position, applying a second set of transformations to the respective spatially transformed input vector at the position to generate a respective output vector at the position” are
mathematical calculations that falls within the mathematical concepts grouping of abstract idea.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 1 recites additional element of “the system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers” which is recited at the very high level of generality such that it amounts no more than mere instructions to apply the exception using generic computer components.
The additional element of “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
The additional element of “a neural network configured to perform the machine learning task” amounts no more than using generic computer with generic neural network to apply the abstract idea.
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 1 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the additional element of “one or more computers and one or more storage devices” and “neural network” are at best the equivalent of merely adding the words “apply it” to the exception. The additional limitation of “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” is mere data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 1 therefore is ineligible.
Claim 2 recites additional element of “ wherein the neural network further comprises: one or more output layers configured to process one or more of the respective output vectors in the output sequence for a last block of the plurality of blocks to generate the network output” amounts no more than using generic computer with generic neural network to apply the abstract idea and at best the equivalent of merely adding the words “apply it” to the exception. claim 2 therefore is ineligible.
Claim 3 recites additional element of “ wherein applying the spatial transformation comprises: for each position, generating a respective first partial vector that includes a first subset of the second number of channels of the respective transformed input vector for the position and a respective second partial vector that includes a second subset of the second number of channels of the respective transformed input vector for the position; applying a normalization to the respective first partial vectors to generate a respective normalized first partial vector for each position; applying, to the respective normalized first partial vectors, a feedforward spatial transformation that combines information across the respective normalized first partial vectors at the positions to generate a respective spatially transformed partial vector for each position; and generating the respective spatially transformed input vector at each of the positions from at least the respective spatially transformed partial vectors and the respective second partial vectors” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 3 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 3 is not patent eligible.
Claim 4 recites additional elements of “ wherein applying, to the respective normalized first partial vectors, a feedforward spatial transformation comprises: determining a product between (i) a spatial transformation matrix and (ii) a matrix of the respective normalized first partial vectors; and adding a bias term to the product to generate the respective spatially transformed partial vectors for each position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 4 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 4 is not patent eligible.
Claim 5 recites additional elements of “wherein generating the respective spatially transformed input vector at each of the positions comprises, for each position: determining an element-wise product between (i) the respective spatially transformed partial vector for the position and (ii) the respective second partial vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 5 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 5 is not patent eligible.
Claim 6 recites additional elements of “wherein generating the respective spatially transformed input vector at each of the positions comprises: applying a self-attention mechanism to the input sequence for the block to generate a respective attended input vector at each of the positions; and for each position: determining a sum between (i) the respective spatially transformed partial vector for the position and (ii) the respective attended input vector for the position to generate a respective combined vector for the position; and determining an element-wise product between (i) the respective combined vector for the position and (ii) the respective second partial vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 6 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 6 is not patent eligible.
Claim 7 recites additional elements of “wherein applying the spatial transformation comprises: applying a normalization to the respective transformed input vectors to generate a respective normalized transformed input vector for each position; applying, to the respective transformed input vectors, a feedforward spatial transformation that combines information across the respective transformed input vectors at the positions to generate a respective spatially transformed vector for each position; and generating the respective spatially transformed input vector at each of the positions from at least the respective spatially transformed vectors and the respective transformed input vectors” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 7 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 7 is not patent eligible.
Claim 8 recites additional elements of “wherein applying, to the respective transformed input vectors, a feedforward spatial transformation comprises: determining a product between (i) a spatial transformation matrix and (ii) a matrix of the respective transformed input vectors; and adding a bias term to the product to generate the respective spatially transformed vectors for each position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 8 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 8 is not patent eligible.
Claim 9 recites additional elements of “wherein generating the respective spatially transformed input vector at each of the positions comprises, for each position: determining an element-wise product between (i) the respective spatially transformed vector for the position and (ii) the respective transformed input vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 9 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 9 is not patent eligible.
Claim 10 recites additional elements of “wherein generating the respective spatially transformed input vector at each of the positions comprises: applying a self-attention mechanism to the input sequence for the block to generate a respective attended input vector at each of the positions; and for each position: determining a sum between (i) the respective spatially transformed vector for the position and (ii) the respective attended input vector for the position to generate a respective combined vector for the position; and determining an element-wise product between (i) the respective combined vector for the position and (ii) the respective transformed input vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 10 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 10 is not patent eligible.
Claim 11 recites additional elements of “wherein for each position, applying a first set of transformations to the respective input vector at the position comprises: applying a normalization to the respective input vectors to generate a respective normalized input vector for each position; for each position, applying a first projection matrix to the respective normalized input at the position to generate a respective initial transformed input vector for the position having the second number of channels; and for each position, applying an activation function to the respective initial transformed input vector for the position to generate the respective transformed input vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 11 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 11 is not patent eligible.
Claim 12 recites additional elements of “wherein for each position, applying a second set of transformations to the respective input vector at the position comprises: applying a second projection matrix to the respective spatially transformed input vector at the position to generate a respective initial output vector for the position having the first number of channels” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 12 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 12 is not patent eligible.
Claim 13 recites additional elements of “wherein for each position, applying a second set of transformations to the respective input vector at the position comprises: adding the respective initial output vector for the position to the respective input vector to the position to generate the respective output vector for the position” which are mathematical calculations that fall within the mathematical concepts grouping of abstract idea. Claim 13 does not include any additional element that integrates the abstract idea into practical application in step 2A-Prong 2 and amounts to significantly more than the judicial exception in step 2B. Claim 13 is not patent eligible.
Claim 14 recites additional elements of “wherein the input sequence for a first block of the plurality of blocks is a sequence of embeddings that represent the network input” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)). Data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 14 therefore is ineligible.
Claim 15 recites additional elements of “wherein the embeddings are not generated using any positional embeddings” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)). Data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 15 therefore is ineligible.
Claim 16 recites additional elements of “wherein the network input is an image, and wherein the sequence of embeddings comprises a respective embedding representing each of a plurality of patches from the image” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)). Data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 16 therefore is ineligible.
Claim 17 recites additional elements of system of claim 1, wherein the machine learning task operates on a network input that is an input sequence to generate a network output for the network input, and the machine learning task comprises: an audio processing task, wherein the network input is a sequence representing a spoken utterance, and the network output is a classification output that classifies the spoken utterance into one or more categories from a set of categories; a health prediction task, wherein the network input is a sequence derived from electronic health record data for a patient, and the network output is a predicted diagnosis for the patient; an agent control task, wherein the network input is a sequence of observations or other data characterizing states of an environment, and the network output defines an action to be performed by the agent in response to the most recent data in the sequence; a genomics task, wherein the network input is a sequence representing a fragment of a DNA sequence or other molecule sequence, and the network output is either an embedding of the fragment for use in a downstream task or an output for the downstream task; or a computer vision task, wherein the network input is an image or a point cloud and the output is a computer vision output for the image or point cloud” . These additional elements amount no more than using generic computer with generic neural network to apply the abstract idea and at best the equivalent of merely adding the words “apply it” to the exception. claim 17 therefore is ineligible.
Claim 18 recites additional elements of system of claim 1 ”wherein the machine learning task is an image classification task, the network input is an image, and the network output is a classification output that includes a respective score for each of a plurality of categories, with each score representing the likelihood that the image includes an object belonging to the category”. These additional elements amount no more than using generic computer with generic image classification to apply the abstract idea and at best the equivalent of merely adding the words “apply it” to the exception. claim 18 therefore is ineligible.
Claim 19:
Step 1: Statutory Category ?: Yes. claim 19 recites one or more computer readable storage media (i.e., an article of manufacture) which is statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitations: “for each position, applying a first set of transformations to the respective input vector at the position to generate a respective transformed input vector at the position, each respective transformed input vector having a second number of channels; generating a respective spatially transformed input vector at each of the positions, comprising applying a feedforward spatial transformation that integrates information across the plurality of positions; and generating an output sequence for the block by, for each position, applying a second set of transformations to the respective spatially transformed input vector at the position to generate a respective output vector at the position” are
mathematical calculations that falls within the mathematical concepts grouping of abstract idea.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 19 recites additional element of “one or more computer readable storage media” which is recited at the very high level of generality such that it amounts no more than mere instructions to apply the exception using generic computer components.
The additional element of “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
The additional element of “a neural network configured to perform the machine learning task” amounts no more than using generic computer with generic neural network to apply the abstract idea.
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 19 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the additional element of “one or more computer readable storage media” and “neural network” are at best the equivalent of merely adding the words “apply it” to the exception. The additional limitation of “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” is mere data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 19 therefore is ineligible.
Claim 20:
Step 1: Statutory Category ?: Yes. claim 20 recites a method (i.e., a process) which is statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
The limitations: “for each position, applying a first set of transformations to the respective input vector at the position to generate a respective transformed input vector at the position, each respective transformed input vector having a second number of channels; generating a respective spatially transformed input vector at each of the positions, comprising applying a feedforward spatial transformation that integrates information across the plurality of positions; and generating an output sequence for the block by, for each position, applying a second set of transformations to the respective spatially transformed input vector at the position to generate a respective output vector at the position” are
mathematical calculations that falls within the mathematical concepts grouping of abstract idea.
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 20 recites additional element of “one or more computer” which is recited at the very high level of generality such that it amounts no more than mere instructions to apply the exception using generic computer.
The additional element of “receiving input” and “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
The additional element of “a neural network configured to perform the machine learning task” amounts no more than using generic computer with generic neural network to apply the abstract idea.
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 20 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the additional element of “one or more computer” and “neural network” are at best the equivalent of merely adding the words “apply it” to the exception. The additional limitation of “receiving input” and “obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions” are mere data gathering and is well-understood, routine conventional activities previously known to the industry and therefore does not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)), subsection II. Even when considered in combination, the additional elements do not provide an inventive concept, claim 20 therefore is ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2 ,7 14-17 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shazeer et al.(WO/2019/084551 , hereinafter “Shazeer” ) (Cited on Applicant’s IDS filed 4/18/2023)
As to claim 1, Shazeer teaches a system for performing a machine learning task on a network input to generate a network output, the system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers (Shazeer page 4, lines 28-30 teaches neural network system 100) , cause the one or more computers to implement:
a neural network configured to perform the machine learning task (Shazeer page 5, lines 6-10 teaches neural network system can perform variety of tasks), the neural network comprising a plurality of blocks, each block configured to perform operations comprising:
obtaining an input sequence for the block comprising a respective input vector at each of a plurality of positions, each input vector having a first number of channels; (Shazeer page 7, lines 20-25 teaches each decoder subnetworks 130 is configured to receive a respective decoder subnetwork input for each of the plurality of combined sequence position and to generate a respective subnetwork output for each of the plurality of combined sequence positions )
for each position, applying a first set of transformations to the respective input vector at the position to generate a respective transformed input vector at the position, each respective transformed input vector having a second number of channels (Shazeer page 7, lines 23-25 teaches the decoder subnetwork outputs generated by the last decoder subnetwork in the sequence are then provided as input to the linear layer 180.Shazeer page 11, lines 25-34 teaches transformations applied by the sublayer reduce the dimensionality of the original keys and values and optionally, the queries);
generating a respective spatially transformed input vector at each of the positions, comprising applying a feedforward spatial transformation that integrates information across the plurality of positions.(Shazeer page 8, lines 1-5 teaches applying an attention mechanism over the inputs at the combined sequence positions preceding the corresponding position using one or more queries derived from the input at the particular output position to generate a updated representation for the particular output position. Shazeer page 8, lines 25-30 teaches some or all of decoder subnetworks can also include a position-wise feed forward layer that is configured to operate on each position in the combined sequence separately)
generating an output sequence for the block by , for each position, applying a second set of transformations to the respective spatially transformed input vector at the position to generate a respective output vector at the position.( Shazeer page 8, lines 25-34 teaches for each combined sequence position, the feed-forward layer 134 is configured to receive an input at the combined sequence position and apply a sequence of transformations to the input at the combined position to generate an output for the combined sequence position. The sequence of transformations can include two or more learned linear transformations each separated by an activation function)
As to claim 2, Shazeer teaches the system of claim 1, wherein the neural network further comprises: one or more output layers configured to process one or more of the respective output vectors in the output sequence for a last block of the plurality of blocks to generate the network output.(Shazeer page 7, lines 25-30 [0046] teaches the decoder subnetwork outputs generated by the last decoder subnetwork in the sequence are then provided as input to the linear layer 180)
As to claim 7, Shazeer teaches the system of claim 1, wherein applying the spatial transformation comprises: applying a normalization to the respective transformed input vectors to generate a respective normalized transformed input vector for each position (Shazeer page 9, lines 5-12 teaches normalization layer that applies layer normalization to the decoder self-attention residual output);
applying, to the respective transformed input vectors, a feedforward spatial transformation that combines information across the respective transformed input vectors at the positions to generate a respective spatially transformed vector for each position (Shazeer page 9, lines 5-12 teaches In cases where a decoder subnetwork can also include a residual connection layer that combines the outputs of the position-wise feedforward layer with inputs to the position -wise feed forward layer to generate a decoder position -wise residual output); and generating the respective spatially transformed input vector at each of the positions from at least the respective spatially transformed vectors and the respective transformed input vectors. ( Shazeer page 9, lines 13-19 teaches the linear layer 180 applies a learned linear transformation to the output of the last decoder subnetwork)
As to claim 14, Shazeer teaches the system of claim 1, wherein the input sequence for a first block of the plurality of blocks is a sequence of embeddings that represent the network input.(See Shazeer Fig. 1, input sequence 102)
As to claim 15, Shazeer teaches the system of claim 14, wherein the embeddings are not generated using any positional embeddings.(Shazeer page 6 , lines 27-32 teaches the embedding layer 120 is configured to , for each token in the combined sequence, map the token to a numeric representation of the token in an embedding space, such as into a vector in the embedding space)
As to claim 16, Shazeer teaches the system of claim 14, wherein the network input is an image, and wherein the sequence of embeddings comprises a respective embedding representing each of a plurality of patches from the image.(Shazeer page 4, lines 17-25 teaches the system may be part of an image processing system . The input sequence can be an image such as a sequence of color values form the image)
As to claim 17, Shazeer teaches system of claim 1, wherein the machine learning task operates on a network input that is an input sequence to generate a network output for the network input (Shazeer page 3 teaches generating a target sequence that includes a respective output at each of multiple positions in an output order from an input sequence), and the machine learning task comprises:
an audio processing task, wherein the network input is a sequence representing a spoken utterance, and the network output is a classification output that classifies the spoken utterance into one or more categories from a set of categories (Shazeer page 4 , lines 1-5 teaches the system may be a speech recognition system. That is , if the input sequence is a sequence of audio data representing a spoken utterance, the target sequence may be a sequence of graphemes, characters or words that represent the utterance)
a health prediction task, wherein the network input is a sequence derived from electronic health record data for a patient, and the network output is a predicted diagnosis for the patient; an agent control task, wherein the network input is a sequence of observations or other data characterizing states of an environment, and the network output defines an action to be performed by the agent in response to the most recent data in the sequence; a genomics task, wherein the network input is a sequence representing a fragment of a DNA sequence or other molecule sequence, and the network output is either an embedding of the fragment for use in a downstream task or an output for the downstream task; or a computer vision task, wherein the network input is an image or a point cloud and the output is a computer vision output for the image or point cloud.
Claims 19-20 merely recite a one or more computer readable storage media storing instructions and a method performed by the system of claim 1. Accordingly, Shazeer teaches every limitation of claims 19-20 as indicates in the above rejection of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Shazeer and further in view of Geva et al. US Patent Application Publication 2020/0193299 A1, hereinafter “Geva”)
As to claim 11, Shazeer teaches the system of claim 1, wherein for each position, applying a first set of transformations to the respective input vector at the position comprises: applying a normalization to the respective input vectors to generate a respective normalized input vector for each position; (Shazeer page 8, lines 19-25 teaches normalization layer that applies layer normalization to the decoder self-attention residual output)
[ for each position, applying a first projection matrix to the respective normalized input at the position to generate a respective initial transformed input vector for the position having the second number of channels]; and for each position, applying an activation function to the respective initial transformed input vector for the position to generate the respective transformed input vector for the position. (Shazeer page 9, lines 13-20 teaches the linear layer applies a learned linear transformation to the output of the last decoder subnetwork 130 in order to project the output of the last decoder subnetwork into the appropriate space for processing the SoftMax layer)
Shazeer fails to expressly teach for each position, applying a first projection matrix to the respective normalized input at the position to generate a respective initial transformed input vector for the position having the second number of channels.
However, Geva teaches applying a first projection matrix to the respective normalized input at the position to generate a respective initial transformed input vector for the position having the second number of channels (Geva par [0085] teaches these two separate applications of PCA provide a projection matrix which can be used to reduce the dimensions of each channel , thereby providing a data matrix of reduced dimensionality)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaching of Shazeer and Geva to achieve the claimed invention. One would have been motivated to make such combination to provide data matrix of reduced dimensionality. (Geva par [0085])
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Shazeer and further in view of Yuan et al.(US Patent Application Publication 2022/0027784 A1, hereinafter “Yuan”)
As to claim 12, Shazeer teaches the system of claim 1 wherein for each position, applying a second set of transformations to the respective input vector at the position comprises: applying a second projection matrix to the respective spatially transformed input vector at the position to generate a respective initial output vector for the position having the first number of channels. (Shazeer page 8, lines 25-34 teaches for each combined sequence position, the feed-forward layer 134 is configured to receive an input at the combined sequence position and apply a sequence of transformations to the input at the combined position to generate an output for the combined sequence position)
Shazeer fails to expressly teach a second projection matrix.
However, Yan teaches a second projection matrix. (Yuan par [0026] teaches inverse matrix)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaching of Shazeer and Yaun to achieve the claimed invention. One would have been motivated to make such combination to increase the performance , accuracy , and / or effectiveness of the new machine learning model (Yuan par [0017])
As to claim 13, Shazeer and Yuan teach the system of claim 12, wherein for each position, applying a second set of transformations to the respective input vector at the position comprises: adding the respective initial output vector for the position to the respective input vector to the position to generate the respective output vector for the position.( Shazeer page 9, lines 5-10 teaches the decoder subnetwork can also include a residual connection layer that combines the outputs of the position wise feedforward layer to generate a decoder position wise residual output)
Claim10 is rejected under 35 U.S.C. 103 as being unpatentable over Shazeer and further in view of Abraham.(US Patent Application Publication 2021/0217443 A1, hereinafter “Abraham”)
As to claim 18, Shazeer teaches the system of claim 1 but fails to teach wherein the machine learning task is an image classification task, the network input is an image, and the network output is a classification output that includes a respective score for each of a plurality of categories, with each score representing the likelihood that the image includes an object belonging to the category.
However, Abraham teaches wherein the machine learning task is an image classification task, the network input is an image, and the network output is a classification output that includes a respective score for each of a plurality of categories, with each score representing the likelihood that the image includes an object belonging to the category. (Abraham par [0041] teaches the style transfer neural network can generate a classification for the segmented video fame using a trained image classification neural network that is configured to process an image to generate a classification output that includes a respective score corresponding to teach of multiple categories)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaching of Shazeer and Abraham to achieve the claimed invention. One would have been motivated to make such combination to automate the process and generate a final stylized video significantly more quickly. (Abraham par [0018])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HIEN DUONG whose telephone number is (571)270-7335. The examiner can normally be reached Monday-Friday 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HIEN L DUONG/Primary Examiner, Art Unit 2147