DETAILED ACTION
1. This office action is in response to the Application No. 18737906 filed on 04/06/2026. Claims 1-18 are presented for examination and are currently pending. Applicant’s arguments have been carefully and respectfully considered.
Response to Arguments
2. Upon review of the claim amendment filed 04/06/2026, the 112 rejection has been withdrawn.
Applicant’s arguments are moot in view of the new grounds of rejection. The examiner is withdrawing the 103 rejections in the previous office action because the Applicant’s amendments necessitated the new grounds of rejection in this office action.
It is noted that Jiang, the newly added primary reference in view of Im now teaches independent claims 1 and 7, Jiang in view of Im in view of Isik now teaches independent claim 13.
Claim Objections
3. Claim 1 are objected to because of the following informalities:
In claim 1, the limitation recites “generate a plurality of latent space vectors vectors by processing ...”. It should be “generate a plurality of latent space vectors by processing ...”.
In claims 1, 7 and 13, the limitation “wherein each of the plurality of output vectors comprises at least one new element extending the corresponding input vector” is a repeated limitation. This limitation should be removed.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1 and 7 are rejected under 35 U.S.C 103 as being unpatentable over
Jiang et al. ("Transformer VAE: A hierarchical model for structure-aware and interpretable music representation learning." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020) in view of Im et al. (US20230169709)
Regarding claim 1, A deep learning system with a latent transformer core (Fig. 1: An overview of the model, pg. 517, right col.,; we designed the Transformer Variational AutoEncoder, abstract)
receive a plurality of input vectors (x1, x2, x3, when T = 3, Fig. 3; Formally, let x1..T denotes the original melody with T bars (measures) in length, where xi represents the melody of the i-th bar, pg. 517, left col., second para., Fig. 1);
generate a plurality of latent space vectors (As an example, we choose two 8-bar sample songs x1(1)..T, and x1 (2)..T as shown in Fig. 5 and encode them into the latent representation z(1) 1..T and z(2) 1..T. pg. 519. right col., first para.) by processing the plurality of input vectors (x1, x2, x3, when T=3, Fig. 3; Formally, let x1..T denotes the original melody with T bars (measures) in length, where xi represents the melody of the i-th bar, pg. 517, left col., second para., Fig. 1) through a variational autoencoder's encoder (CVAE Encoder (conditional Variational Autoencoder), Fig. 3; A conditional VAE view of the Transformer VAE when T=3, Fig. 3, pg. 518, left col.; x0...T-1→Local encoder, Fig. 1; x3→CVAE Encoder, Fig. 3),
wherein the variational autoencoder's encoder comprises a plurality of network layers of successively smaller sizes, each successive layer receiving as input a vector smaller than the input to the preceding layer (CVAE Encoder, Fig. 3; Local Encoder in Fig. 1),
the last of which outputs the plurality of latent space vectors (latent representation z(1) 1..T and z(2) 1..T. pg. 519. right col., first para.), each of the plurality of latent space vectors having a size determined by the last layer of the variational autoencoder's encoder (Examiner notes the Encoder in Figs. 1 and 3 comprises a plurality of network layers of successively smaller sizes which is similar to VAE Encoder Subsystem 150 of the instant’s specification’s Fig. 1C);
learn relationships between the plurality of latent space vectors by processing the plurality of latent space vectors through a transformer (We implemented the Transformer VAE model with a latent dimension dim(zi) =64 for each bar. The Transformer encoder (decoder) consists of N=3 stacked identical encoder(decoder) layers, pg. 518, right col., last para.),
wherein the transformer comprises a plurality of multihead attention blocks (Masked Multi-Head Self Attention blocks and Masked Multi-Head Inter Attention blocks, Fig. 1) each configured to receive input vectors of the same size as the plurality of latent space vectors (We made some changes to make the vanilla Transformer model capable of representation learning under a VAE set ting, i.e., the model input and output are identical, pg. 517, left col., first para.), and
wherein successive ones of the plurality of latent space vectors are received as successive inputs to the transformer, and wherein outputs of an encoder of the transformer are provided as inputs to a decoder of the transformer (Then, the Transformer encoder takes he1..T as inputs and calculate the parameters of the latent code z: ... The latent variable z is then fed into the Transformer decoder, pg. 517, left col., second para.);
use the learned relationships between the plurality of latent space vectors to generate, using the decoder of the transformer, a plurality of output latent space vectors based on the plurality of input vectors (The latent variable z is then fed into the Transformer decoder DTransformer, together with the representation of previ ously decoded bars to decode the new bars sequentially, pg. 517, left col., second para.); and
generate output vectors (where the ˆx1..T are the decoded bars, pg. Fig. 1, pg. 517, left col., second para.) by passing the plurality of output latent space vectors through the variational autoencoder's decoder (CVAE Decoder, Fig. 3; Local decoder, Fig. 1),
wherein the variational autoencoder's decoder comprises a plurality of network layers of successively larger sizes (CVAE Decoder, Fig. 3; Local decoder, Fig. 1. The Examiner notes that the Decoder in Fig. 1 and 3 comprises a plurality of network layers of successively larger sizes (which is similar to the VAE Decoder Subsytem 180 of instant specification’s Fig. 1C)),
a first layer of the variational autoencoder's decoder receiving as input vectors of the same size as the plurality of latent space vectors (We made some changes to make the vanilla Transformer model capable of representation learning under a VAE set ting, i.e., the model input and output are identical, pg. 517, left col., first para.);
wherein each of the plurality of output vectors (the decoder will help the 5th bar fetch back the information from the 1st bar and complete the reconstruction, pg. 517, left col., last para. The Examiner notes the decoder reconstructed the input) comprises at least one new element extending the corresponding input vector (if we change the context of some bars, we expect the whole reconstructed music to be changed accordingly, pg. 517, left col., last para.);
Jiang does not explicitly teach a deep learning system with a latent transformer core for large codeword models, comprising one or more computers with executable instructions that, when executed, cause the deep learning system to:
Im teaches a deep learning system with a latent transformer core (Among deep learning language models, the BERT model currently shows good performance in the word prediction field. The BERT model is a model designed using an encoder part of a transformer structure [0063])
for large codeword models (generating a codebook to represent the plurality of pieces of facial image training data with block codebook indices [0010]), comprising one or more computers with executable instructions that, when executed, cause the deep learning system to (The scope of the present disclosure may include software or machine-executable instructions (for example, an operation system (OS), applications, firmware, programs, etc.), which enable operations of a method according to various embodiments to be executed in a device or a computer, and a non-transitory computer-readable medium capable of being executed in a device or a computer each storing the software or the instructions [0099]):
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Jiang to incorporate the teachings of Im for the benefit of generating an image generation model using a bidirectional encoder representations from transformers (BERT) model that covers some tokens with a mask among the codebook indices in the facial image training data 10′ represented with the codebook indices and predicts what are the tokens covered with the mask (Im [0062])
Regarding claim 7, claim 7 is similar to claim 1. It is rejected in the same manner and reasoning applying.
5. Claims 2-4 and 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al. ("Transformer vae: A hierarchical model for structure-aware and interpretable music representation learning." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020) in view of Im et al. (US20230169709) and further in view of Saber et al. (US20230131694)
Regarding claim 2, Modified Jiang teaches the system of claim 1, Modified Jiang does not explicitly teach the limitation s of claim 2.
Saber teaches wherein the input vectors may contain a plurality of appended zeros (The matrices input to the encoder may be reshaped to have a fixed size by appending zeros in a configuration that may be commonly understood between a UE and a gNB [0178]) and
a plurality of truncated data points may be used to train (In any of the embodiments of frameworks disclosed herein, a quantizer function may be differentiable with a derivative value of essentially zero (e.g., with a probability of 1) throughout some or all of a quantizer range (e.g., essentially throughout the entire range) [0102]. The Examiner notes truncation is a type of quantization) and
operationalize a transformer that predicts the next sequential vector following an input vector (Thus, a reconstructed input may be the input applied to the generation model, or an approximation, estimate, prediction, etc., of the input applied to the generation model [0028]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Jiang to incorporate the teachings of Saber for the benefit of a generation model 303 which may be trained to generate a feature vector that may identify or separate one or more features (e.g., latent features) of the training data that may reduce the overhead associated with storing and/or transmitting the representation 307 (Saber [0064])
Regarding claim 3, Modified Jiang teaches the system of claim 1, Modified Jiang does not explicitly teach the limitation s of claim 3.
Saber teaches wherein the input vectors may contain a plurality of appended metadata (Thus, a training set may originally include matrices of different sizes which may be modified by appending zeros as described above to convert the matrices to one fixed matrix size [0178]; CSI matrices from time from ti-1 to time ti. The Examiner notes CSI matrices from time from ti-1 to time ti relates to temporal information).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Jiang to incorporate the teachings of Saber for the benefit of a generation model 303 which may be trained to generate a feature vector that may identify or separate one or more features (e.g., latent features) of the training data that may reduce the overhead associated with storing and/or transmitting the representation 307 (Saber [0064])
Regarding claim 4, Modified Jiang teaches the system of claim 3, Saber teaches wherein metadata comprises data type, temporal information, data source, data characteristics, and domain-specific metadata (Thus, a training set may originally include matrices of different sizes which may be modified by appending zeros as described above to convert the matrices to one fixed matrix size [0178]; CSI matrices from time from ti-1 to time ti. The Examiner notes CSI matrices from time from ti-1 to time ti relates to temporal information).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Jiang to incorporate the teachings of Saber for the benefit of a generation model 303 which may be trained to generate a feature vector that may identify or separate one or more features (e.g., latent features) of the training data that may reduce the overhead associated with storing and/or transmitting the representation 307 (Saber [0064])
Regarding claim 8, claim 8 is similar to claim 2. It is rejected in the same manner and reasoning applying.
Regarding claim 9, claim 9 is similar to claim 3. It is rejected in the same manner and reasoning applying.
Regarding claim 10, claim 10 is similar to claim 4. It is rejected in the same manner and reasoning applying.
6. Claims 5, 6, 11-13, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al. ("Transformer vae: A hierarchical model for structure-aware and interpretable music representation learning." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020) in view of Im et al. (US20230169709) and further in view of Isik et al. (US20240195438)
Regarding claim 5, Modified Jiang teaches the system of claim 1, Im teaches a plurality of codewords (The facial area generation operation may include a codebook training operation of training and generating a codebook to represent the plurality of pieces of facial image training data with block codebook indices [0010]. The Examiner notes a codebook consist of plurality of codewords).
The same motivation to combine dependent claim 1 applies here.
Modified Jiang does not explicitly teach wherein the plurality of input vectors comprises a plurality of codewords.
Isik teaches wherein the plurality of input vectors comprises a plurality of codewords (input vector 105 to each codeword 115 a-n [0024]. The Examiner notes codeword 115 a-n is an input in Fig. 1).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Jiang to incorporate the teachings of Isik for the benefit of using attention layers in Transformer networks [0024] to generate compressed data, where the compressed data includes an indication of the predetermined number of codewords of the plurality of codewords (Isik [0012])
Regarding claim 6, Modified Jiang teaches the system of claim 5, Isik teaches wherein the plurality of codewords are converted into the plurality of latent space vectors (The distance vector obtained may then be converted to a probability distribution over the codebook for each input vector [0025]).
The same motivation to combine dependent claim 5 applies here.
Regarding claim 11, claim 11 is similar to claim 5. It is rejected in the same manner and reasoning applying.
Regarding claim 12, claim 12 is similar to claim 6. It is rejected in the same manner and reasoning applying.
Regarding claim 13, claim 13 is similar to claim 1. It is rejected in the same manner and reasoning applying. Im teaches computer-readable storage media having computer-executable instructions embodied thereon that, when executed by one or more processors of a computing system (The scope of the present disclosure may include software or machine-executable instructions (for example, an operation system (OS), applications, firmware, programs, etc.), which enable operations of a method according to various embodiments to be executed in a device or a computer, and a non-transitory computer-readable medium capable of being executed in a device or a computer each storing the software or the instructions [0099])
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Jiang to incorporate the teachings of Im for the benefit of generating an image generation model using a bidirectional encoder representations from transformers (BERT) model that covers some tokens with a mask among the codebook indices in the facial image training data 10′ represented with the codebook indices and predicts what are the tokens covered with the mask (Im [0062])
Modified Jiang does not explicitly teach a non-transitory, computer-readable storage media having computer-executable instructions embodied thereon that, when executed by one or more processors of a computing system employing an asset registry platform,
Isik teaches a non-transitory, computer-readable storage media having computer-executable instructions embodied thereon that, when executed by one or more processors of a computing system (processor(s) 802 is/are at least one hardware processor configured to execute various tasks, operations and/or functions for computing device 800 as described herein according to software and/or instructions configured for computing device 800 [0064]) employing an asset registry platform (Data/information being tracked and/or sent to one or more entities as discussed herein could be provided in any database, table, register, …, storage, and/or storage structure [0071])
for a latent transformer core for a Large Codeword Model (Instead of selecting a single codeword from the codebook to approximately represent the input vector, a weighted combination of multiple codewords is used [0022]; attention layers in Transformer networks [0024]; According to other examples, the disclosed techniques may be implemented in systems in which the same encoder and decoder are used [0030]),
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Modified Jiang to incorporate the teachings of Isik for the benefit of using attention layers in Transformer networks [0024] to generate compressed data, where the compressed data includes an indication of the predetermined number of codewords of the plurality of codewords (Isik [0012])
Regarding claim 17, claim 17 is similar to claim 5. It is rejected in the same manner and reasoning applying.
Regarding claim 18, claim 18 is similar to claim 6. It is rejected in the same manner and reasoning applying.
7. Claims 14-16 is rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al. ("Transformer vae: A hierarchical model for structure-aware and interpretable music representation learning." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020) in view of Im et al. (US20230169709) in view of Isik et al. (US20240195438) and further in view of Saber et al. (US20230131694)
Regarding claim 14, claim 14 is similar to claim 2. It is rejected in the same manner and reasoning applying.
Regarding claim 15, claim 15 is similar to claim 3. It is rejected in the same manner and reasoning applying.
Regarding claim 16, claim 16 is similar to claim 4. It is rejected in the same manner and reasoning applying.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MORIAM MOSUNMOLA GODO whose telephone number is (571)272-8670. The examiner can normally be reached Monday-Friday 8am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached on (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.G./Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/ Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights