DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1,2,4 and 7-12, 14 and 17-19 are rejected under 35 U.S.C. 102(a) as being anticipated by US20210049990 (Medeot), hereinafter US’990.
Regarding claim 1, US’990 discloses ‘A musical score creation device (US’990, ¶[0279],¶[0280],¶[0057]:”the outputs may be musical notes, either played or stored as data, each note having its own time instant. At least some of the musical notes are generated”) comprising: at least one processor (US’990, ¶[0047]:” computer system is provided comprising a data processing stage in the form of one or more processors, such as CPUs”) configured to execute a receiving unit configured to receive a note sequence that includes a plurality of musical notes (US’990, ¶[0058]:”methods disclosed herein allow music data, corresponding to musical notes, to be generated … probabilities are used to choose notes in a sequence, eventually arriving at a complete sequence of notes that can be regarded as a piece of music”),
and an estimation unit configured to, by using a trained model, estimate each note (US’990, ¶¶[0085-[0087]:“A neural network is trained on pre-existing data and then , once trained , may be used to provide an appropriate output based on a new data input”; ”… the structure neural network is first trained using training data. … the training data is music data since the structure neural network is being used to determine one or more probabilities relating to musical notes”) and attribute information for creating a musical score (US’990, ¶¶[0092]-[0094]:”… parameters are note type, note duration, note pitch, note interval, note position … “parameters and associated values of each note in the music data”)
the trained model being a machine-learning model that has learned an input-output relationship between a reference note sequence including a plurality of reference notes (US’990, ¶[0065]:“…PSMs include Markov chains , probabilistic grammars , and recurrent neural networks with a probabilistic final layer ( SOFTMAX etc.)”), and each reference note and reference attribute information for creating a reference musical score (US’990, ¶¶[0085]-[0087]:”The training data is converted into a plurality of training vectors … and the training vectors are then processed by the structure neural network in order to train it”).
Regarding claim 2, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 further discloses ‘wherein the at least one processor is further configured to execute a generation unit configured to generate musical score information indicating the musical score that describes each note and the attribute information that have been estimated (US’990, ¶[0087]:”FIG. 2 illustrates a measure or bar of music 201 comprising four crotchets 202 to 205, which is an example of music data represented visually using conventional modern musical notation”).
Regarding claim 4, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 further discloses ‘wherein the estimation unit (¶¶[0092]-[0094]) is configured to estimate division and joining of note values as the attribute information (US’990, ¶¶[0127]-[0128]:“a maximum note duration may be set, and any notes longer than this may be split into multiple notes of the same pitch. For example, the maximum note duration may be two beats”).
Regarding claim 7, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 further discloses ‘wherein the at least one processor is further configured to execute a first determination unit configured to determine an accidental based on each note and the attribute information that have been estimated (US’990, ¶[0097]:”any pitch that is not included in the major scale in the key of C major may be approximated to one of the pitches included in the major scale. For example, … F# may be approximated to a pitch class of F”).
Regarding claim 8, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 further discloses ‘wherein the at least one processor is further configured to execute a second determination unit configured to determine a time signature based on each note and the attribute information that have been estimated (US’990, ¶[0146]:” The music data exists in bars of music, where all bars have a fixed bar duration based on the time signature of the music”).
Regarding claim 9, US’990 discloses ‘A musical score creation device comprising: at least one processor (US’990, ¶[0047]:” computer system is provided comprising a data processing stage in the form of one or more processors, such as CPUs”) configured to execute a receiving unit configured to receive an input note token sequence, which is performance data including information on a musical note, a part, a beat, and a bar (US’990, ¶[0066]:”Such a hidden layer/stateful component may be LSTM layer, and such layers have been widely used in RNNs to model … language token sequences and musical sequences”),
an estimation unit configured to estimate a musical score token sequence from the input note token sequence (US’990, ¶[0065]:“…PSMs include Markov chains , probabilistic grammars , and recurrent neural networks with a probabilistic final layer ( SOFTMAX etc.)”), by using a trained model that has been trained by using a musical note token sequence for learning as an input and a musical score element token sequence as an output (US’990, ¶[0089]:”FIG. 3 illustrates a flow diagram of a method of generating one or more vectors from music data , the vectors being training vectors used as an input to train a neural network”),
the musical score element token sequence being converted from a reference image musical score and including information on a musical note drawing, an attribute, and a bar, the musical note token sequence for learning being created from the musical score element token sequence (US’990, ¶¶[0092]-[0094]:”Music data is first provided, and … is then processed to determine at least one parameter for each note of the music data, each parameter having a value . .. note type, note duration, note pitch, note interval, note position, …depending on the desired outcome of the training”; ”At step s301, parameters and associated values of each note in the music data are determined, and each note is labelled with the parameter values”, ¶[0087]:“FIG. 2 illustrates a measure or bar of music 201 comprising four crotchets 202 to 205 , which … music data represented visually”),
and a creation unit configured to create an image musical score from the musical score token sequence (US’990, ¶[0087]:”FIG. 2 illustrates a measure or bar of music 201 comprising four crotchets 202 to 205, which is an example of music data represented visually using conventional modern musical notation”).
Regarding claim 10, US’990 discloses ‘A training device comprising: at least one processor configured to execute a first acquisition unit configured to acquire a reference note sequence including a plurality of reference notes, (US’990, ¶¶[0085-[0087]:“A neural network is trained on pre-existing data and then , once trained , may be used to provide an appropriate output based on a new data input”; ”… the structure neural network is first trained using training data. … the training data is music data since the structure neural network is being used to determine one or more probabilities relating to musical notes”):
a second acquisition unit configured to acquire each reference note and reference attribute information for creating a musical score (US’990, ¶¶[0092]-[0094]:”Music data is first provided, and the music data is then processed to determine at least one parameter for each note of the music data, each parameter having a value . Example parameters are note type, note duration, note pitch, note interval, note position, or any other parameter depending on the desired outcome of the training”; ”At step s301, parameters and associated values of each note in the music data are determined, and each note is labelled with the parameter values”, acquiring/determining parameters and associated values for each note; per-note values are generated during training-data preparation),
and a construction unit configured to construct a trained model that has learned an input-output relationship between the reference note sequence, and each reference note and the reference attribute information (US’990,¶[0085]-[0087]:”A neural network is trained on pre - existing data and then , once trained , may be used to provide an appropriate output based on a new data input”; ¶[0087]:”The training data is converted into a plurality of training vectors using the steps of FIG. 3, described below, and the training vectors are then processed by the structure neural network in order to train it”).
Regarding claim 11, US’990 discloses ‘A musical score creation method executed by a computer (US’990, ¶[0047]:”data processing stage in the form of one or more processors …CPUs”), the musical score creation method (US’990, ¶[0057]:”a method of providing one or more outputs at one or more respective time instants. Particularly , the outputs may be musical notes”) comprising: receiving a note sequence including a plurality of musical notes (US’990, ¶[0044]:”receiving the piece of music at a processing stage; processing the piece of music so as to identity therein a plurality of repeating sections, each repeating section being a repeat of an earlier section”);
and estimating each note and attribute information for creating a musical score, by using a trained model,
the trained model being a machine learning model that has learned an input-output relationship between a reference note sequence including a plurality of reference notes, and each reference note and reference attribute information for creating a reference musical score. (Claim 11 corresponds to claim 1)
Regarding claim 12, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
US’990 further discloses ‘further comprising generating musical score information indicating the musical score that describes each note and the attribute information that have been estimated. (Claim 2 corresponds to claim 2)
Regarding claim 14, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
wherein in the estimating, division and joining of note values are estimated as the attribute information. (Claim 14 corresponds to claim 4)
Regarding claim 17, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
US’990 further discloses ‘further comprising determining an accidental based on each note and the attribute information that have been estimated. (Claim 17 corresponds to claim 7)
Regarding claim 18, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
US’990 further discloses ‘further comprising determining a time signature based on each note and the attribute information that have been estimated. (Claim 18 corresponds to claim 8)
Regarding claim 19, US’990 discloses ‘A training method executed by a computer, the training method comprising: acquiring a reference note sequence including a plurality of reference notes (US’990, ¶[0087]:”The training data is converted into a plurality of training vectors using the steps of FIG. 3, described below, and the training vectors are then processed by the structure neural network in order to train it”);
acquiring each reference note and reference attribute information for creating a musical score (US’990, ¶¶[0092]-[0094]:”Music data is first provided, and the music data is then processed to determine at least one parameter for each note of the music data, each parameter having a value . Example parameters are note type, note duration, note pitch, note interval, note position, or any other parameter depending on the desired outcome of the training”; ”At step s301, parameters and associated values of each note in the music data are determined, and each note is labelled with the parameter values”, );
and constructing a trained model that has learned an input-output relationship between the reference note sequence, and each reference note and the reference attribute information (US’990,¶[0085]-[0087]:”A neural network is trained on pre - existing data and then , once trained , may be used to provide an appropriate output based on a new data input”; ¶[0087]:”The training data is converted into a plurality of training vectors using the steps of FIG. 3, described below, and the training vectors are then processed by the structure neural network in order to train it”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3,6,13 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over US’990, in view of US20210090535 (Miles), hereinafter US’535.
Regarding claim 3, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 does not expressly disclose ‘wherein the estimation unit is configured to estimate a key signature as the attribute information.
However, US’535 discloses ‘wherein the estimation unit is configured to estimate a key signature as the attribute information (US’535, [0164]:”the determination engine determines ( e.g. , estimates or calculates ) a key for an entire piece of music”);¶[0098]:”Key 462 includes the key that the piece of music is created in … estimated key is calculated … The estimated key may provide the probability of each key along with the key itself , and that probability can be used as an indicator of confidence”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify the music processing system of US’990 to include the trained neural-network determination engine of US’535 that estimates the key musical piece. Both references process symbolic music information using trained machine-learning models, incorporating the key estimation performed by the determination engine of US’535 into the music generation process of US’990 would have been a use of a machine-learning technique with another to provide additional musical information for the generated score and improve the consistency of the resulting notation.
Regarding claim 6, US’990 discloses ‘The musical score creation device according to claim 1,
US’990 does not expressly disclose ‘wherein the estimation unit is configured to estimate voice as the attribute information.
However, US’535 discloses ‘wherein the estimation unit is configured to estimate voice as the attribute information (US’535, ¶[0108],¶[0115]:”Texture 428 is .. distinguished according to the number of voices, or parts, and the relationship between these voices”, “Each voice in the MIDI transcriptions is on its own channel”).
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to modify the trained neural network system of US’990 to further estimate voice as taught by US’535 because US’535 recognizes that voice assignment is a musical attribute that can be determined using a trained machine-learning model during music analysis (¶¶[0108],[0115]). Incorporating voice estimation performed by US’990 would improve the completeness of the generated musical score by enabling the trained model to distinguish respective musical notes, using the machine-learning attribute technique. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 417 (2007).
Regarding claim 16, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
US’990 further discloses ‘wherein in the estimating, voice is estimated as the attribute information. (Claim 16 corresponds to claim 6)
Regarding claim 13, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
wherein in the estimating, a key signature is estimated as the attribute information. (Claim 13 corresponds to claim 3)
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over US’990, in view of US10922993 (Marradi), hereinafter US993’.
Regarding claim 5, US’990 discloses ‘The musical score creation device according to claim 1, as discussed above.
US’990 discloses ‘where the estimation unit is configured to estimate e attribute information (US’990, ¶¶[0085-[0087]:“A neural network is trained on pre-existing data and then , once trained , may be used to provide an appropriate output based on a new data input”; ”… the structure neural network is first trained using training data. … the training data is music data since the structure neural network is being used to determine one or more probabilities relating to musical notes”).
US’990 does not expressly disclose that the estimated attribute information includes a clef.
However US’993 teaches that a clef is a fundamental element of musical notation that defines the pitch range of a staff and assigns notes to the lines and spaces of the staff, thereby determining how the notes are represented in the generated score (US’990, col. 19, lines 23-28:”… the staff clef functions as a designator to assign individual notes to the given lines and/or spaces of the staff”). US’993 recognizes the clef as notation attribute information used in representing musical notes.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to modify the estimation performed by US’990 to include estimation of the clef taught by US’993 because the clef is a conventional notation attribute necessary to correctly represent the pitch range and note placement of the generated music score. Incorporating the known notation attribute into the attribute estimation process would have been a predictable use of a known musical notation element in providing completeness and readability of the generated score.
Regarding claim 15, US’990 discloses ‘The musical score creation method according to claim 11, as discussed above.
wherein in the estimating, a clef is estimated as the attribute information. (Claim 15 corresponds to claim 5)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200202825 (Kolen) teaches using deep learning to recognize musical notation from sheet music images and convert into musical data.
US 20200143286 (Frank) teaches creating and processing tokenized musical score data for machine-learning, including representing musical notation elements and their attributes for training and generation.
US 20200074876 (Jancsy) teaches using a machine-learning model to analyze and generate musical information by learning relationships between musical events, notes and their associated attributes for music generation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICOLE K GILLESPIE whose telephone number is (571)482-4187. The examiner can normally be reached Monday-Friday 7:30-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dedei K Hammond can be reached at (571)270-3819. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICOLE K GILLESPIE/Examiner, Art Unit 2837
/DEDEI K HAMMOND/Supervisory Patent Examiner, Art Unit 2837
Prosecution Insights