DETAILED ACTION
Introduction
1. This office action is in response to Applicant's submission filed on 11/01/2024. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are currently pending and examined below.
Drawings
2. The drawings filed on 11/01/2024 have been accepted and considered by the Examiner.
Information Disclosure Statement
3. The Information Statement (IDS) filed on 12/12/2024 has been accepted/considered and is in compliance with the provisions of 37 CFR 1.97.
Priority
4. The Applicants priority to Greek non-provisional application # 20240100747, filed Oct. 24, 2024, has been accepted and considered in this office action.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) The claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
5. Claims 1, 4, 7-11, 14 and 17-20 are rejected under 35 U.S.C. 102 (a) (1) as being anticipated by Jansen (U.S. Patent Application Publication # 2020/0349921 A1).
With regards to claim 1, Jansen teaches a computer-implemented method comprising obtaining a plurality of training examples, wherein each training example of the plurality of training examples includes a first input, a second input, and an output, wherein the first input is multi-dimensional (Abstract, teaches methods for generating training triplets that can be used to train multidimensional embeddings to represent the semantic content of non-speech sounds present in a corpus of audio recordings. Para 22, teaches first, second, third input and an output in this multidimensional space);
identifying an anchor training example, a positive training example, and a negative training example of the plurality of training examples by applying the second inputs of the anchor training example, positive training example, and negative training example to a mapping function to generate respective mapped inputs (Para 3, teaches obtaining training data that includes a plurality of sound recordings; generating a plurality of training triplets, wherein each training triplet of the plurality of training triplets includes a respective anchor audio segment, a positive audio segment, and a negative audio segment from the plurality of sound recordings, wherein generating the plurality of training triplets comprises performing one of a first triplet sampling operation, a second triplet sampling operation, a third triplet sampling operation, or a fourth triplet sampling operation; mapping each audio segment of each training triplet of the plurality of training triplets to a respective feature vector in an n-dimensional feature space);
determining that a first difference between the respective mapped inputs of the anchor training example and the positive training example is less than a first threshold value (Para 38, teaches that where the distance between the anchor segment feature vector and the positive segment feature vector or the “anchor-positive distance” is less than the distance between the anchor segment feature vector and the negative segment feature vector or the “anchor-negative distance” for a particular training triplet e.g., wherein the anchor-positive distance is less than the anchor-negative distance by more than a threshold amount for the particular training triplet, the mapping could be updated based only on other training triplets);
determining that a second difference between the outputs of the anchor training example and the positive training example is less than a second threshold value (Para 41, teaches other examples wherein the loss term for each triplet could be bounded such that the loss term was zero or some other constant value, when the positive feature vector is closer to the anchor feature vector than the negative feature vector e.g., when the anchor-positive Euclidean distance is less than the anchor-negative Euclidean distance by at least a threshold amount).
determining that a third difference between the respective mapped inputs of the anchor training example and the negative training example is less than the first threshold value (Para 41, further teaches that the loss function returns “0” when a difference between the anchor-positive feature vector distance is less than the anchor-negative feature vector distance by more than a threshold amount, also see equation disclosed therein);
and determining that a fourth difference between the outputs of the anchor training example and the negative training example is greater than the second threshold value (Para 63, teaches that a term of the loss function that corresponds to a particular training triplet is increased by increasing an anchor-positive distance relative to an anchor-negative when the anchor-positive distance is not less than the anchor-negative distance by at least a specified threshold amount);
applying the first and second inputs of the anchor training example, the positive training example, and the negative training example to a machine learning model to determine a contrastive loss (Abstract, teaches that these training triplets can be used with a triplet loss function to train the multidimensional embeddings such that the embeddings can be used to cluster the contents of a corpus of audio recordings, to facilitate a query-by-example lookup from the corpus, to allow a small number of manually-labeled audio recordings to be generalized, or to facilitate some other audio classification task);
and updating the machine learning model based on the contrastive loss (Para 35, teaches that the output of the loss function could then be used to update the mapping e.g., via backpropagation or some other machine learning training algorithm).
With regards to claim 4, Jansen teaches the computer-implemented method of claim 1, wherein updating the machine learning model based on the contrastive loss comprises updating the machine learning model such that predicted outputs of the machine learning model for the anchor training example and positive training example are less different and further such that predicted outputs of the machine learning model for the anchor training example and negative training example are more different (Para 48, teaches that different audio segments samples from the same audio recording should be more alike with respect to classification and/or semantic content than audio segments sampled from different audio recordings. Accordingly, a training triplet of spectrograms could be generated by selecting an anchor audio segment from a first audio recording of a training set of audio recordings. The positive audio segment for the triplet could then be generated by sampling another audio segment from the first audio recording. The negative spectrogram for the triplet could then be generated by sampling an audio segment from a second audio recording that differs from the first audio recording. In order to increase the likelihood that the anchor and positive audio segments represent similar classifications and/or semantic contents, the sampling of the positive audio segment could be restricted to audio segments of the first audio recording that are within a maximum time difference from the anchor audio segment).
With regards to claim 7, Jansen teaches the computer-implemented method of claim 1, further comprising, after updating the machine learning model based on the contrastive loss determining at least one aspect of the first input that the machine learning model relies on less than a threshold amount to generate a model output (Para 5, teaches updating the mapping based on the determined feature vectors such that a loss function is reduced. The loss function includes a sum of a plurality of terms, wherein each term in the plurality of terms corresponds to a respective training triplet in the plurality of training triplets. A term of the loss function that corresponds to a particular training triplet is increased by increasing a first distance relative to a second distance when the first distance is not less than the second distance by at least a specified threshold amount, wherein the first distance is between the feature vector of the anchor audio segment of the particular training triplet and the feature vector of the positive audio segment of the particular training triplet, and wherein the second distance is between the feature vector of the anchor audio segment of the particular training triplet and the feature vector of the negative audio segment of the particular training triplet);
and modifying the machine learning model to exclude use of the determined at least one aspect of the first input to generate the model output (Para 30, teaches that a mapping can be used to generate for each of the spectrograms a respective feature vector in the n-dimensional feature space. The first three spectrograms correspond to respective time segments of audio that include a first kind of semantically-related content e.g., different kinds of bird call, or dog barks, while the latter two spectrograms correspond to respective time segments of audio that include a second kind of semantically-related content e.g., emergency vehicle sirens, that is semantically unrelated to the content of the first three spectrograms).
With regards to claim 8, Jansen teaches the computer-implemented method of claim 1, wherein the mapping function comprises a clustering function (Abstract, teaches that these training triplets can be used with a triplet loss function to train the multidimensional embeddings such that the embeddings can be used to cluster the contents of a corpus of audio recordings, to facilitate a query-by-example lookup from the corpus, to allow a small number of manually-labeled audio recordings to be generalized, or to facilitate some other audio classification task).
With regards to claim 9, Jansen teaches the computer-implemented method of claim 1, wherein the mapping function comprises a nonlinear function that projects a multi-dimensional input to a scalar output (Para 24, teaches obtain a mapping e.g., an artificial neural network, a nonlinear kernel, a machine learning algorithm or structure, or some other linear or nonlinear mapping between segments of audio in an n-dimensional feature space, where proximity within the feature space represents semantic similarity between the contents of the audio segment).
With regards to claim 10, Jansen teaches the computer-implemented method of claim 1, wherein the first input is representative of a first set of variables and a second set of variables, wherein a variable represented by the second input has a causal effect on the first set of variables, the second set of variables, and the output, and wherein the first set of variables has a causal effect on the output that is greater than a causal effect on the output by the second set of variables (Paragraphs 18-29, teach the audio input could include both speech and non-speech sounds. Speech sounds could be removed or included in the recordings. In case speech sounds are included, the mapping may define an n-dimensional semantic space that encodes information about speech sounds e.g., phonemes, words, etc. in addition to non-speech sounds. On the other hand, non-speech sounds can include animal noises, dog bark, sound of wind, emergency vehicle sirens etc. Paragraphs 46-47, teach that weightage of each of these categories of sounds can be controlled, i.e. increased or decreased depending on the final application of the mapping. See equation in para 47).
With regards to claims 11, 14 and 17-19, please see the rejection of claims 1, 4, 7-9 above.
With regards to claim 20, this is a computer readable medium (CRM) claim for the corresponding method claim 1. These two sets of claims are related as method and CRM of using the same, with each claimed CRM element's function corresponding to the claimed method step. Accordingly, claim 20 is similarly rejected under the same rationale as applied above with respect to method claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 2-3, 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Jansen in view of Vangala (U.S. Patent Application Publication # 2022/0245267 A1).
With regards to claim 2, Jansen teaches the computer-implemented method of claim 1, wherein the first input represents digital audio content (See Abstract),
However, Jansen may not explicitly detail the limitation wherein the second input identifies a user account from a set of user accounts. This is taught by Vangala (Para 90, teaches receiving user profile data that uniquely corresponds to an individual user account of the plurality of user accounts);
Jansen and Vangala can be considered as analogous art as they belong to a similar field of endeavor in knowledge extraction methods. It would thus have been obvious to one having ordinary skill in the art to advantageously combine the teachings of Vangala (Use of user account as an input) with those of Jansen (Use of audio input for machine learning-based mapping updating) so as to provide information with specific context (Vangala, para 2) which will aid the mapping process of Jansen.
With regards to claim 3, Jansen teaches the computer-implemented method of claim 2, wherein the first input includes a plurality of features representative of digital audio content (Para 3, teaches mapping each audio segment of each training triplet of the plurality of training triplets to a respective feature vector in an n-dimensional feature space).
With regards to claims 12-13, please see the rejection of claims 2-3 above.
Allowable Subject Matter
7. Claims 5-6, 15-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The prior art of record, alone or in combination, does not currently suggest or teach the invention as outlined in these claims. More detailed reasons for allowance will be outlined as and when the Application proceeds to allowability.
Conclusion
8. The following prior art, made of record but not relied upon, is considered pertinent to applicant's disclosure: Jiang (U.S. Patent Application Publication # 2018/0336888 A1), McCallum (U.S. Patent Application Publication # 2020/0074982 A1). These references are also included in the PTO-892 form attached with this office action.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. If you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). In case you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEERAJ SHARMA whose contact information is given below. The examiner can normally be reached on Monday to Friday 8 am to 5 pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis-Desir can be reached on 571-272-7799 (Direct Phone). The fax number for the organization where this application or proceeding is assigned is 571-273-8300.
/NEERAJ SHARMA/
Primary Examiner, Art Unit 2659
571-270-5487 (Direct Phone)
571-270-6487 (Direct Fax)
neeraj.sharma@uspto.gov (Direct Email)
Prosecution Insights