Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is
submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the
following application and has made the following comments below.
Priority
Acknowledgement is made of Applicant’s claim of the present application being a
continuation of PCT International Patent Application No. PCT/JP/2022/006421, filed on
02/17/2022.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 08/16/2024 and 07/15/2025
are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure
statements are being considered by the examiner.
Amendment
Applicant submitted amendments 07/22/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Status of Claims
Claims 1-20 are pending.
Claims 1, 2, 4-6, and 9-18 are amended.
Claims 19 and 20 are newly added by the present amendment.
Applicants Arguments:
Regarding Argument 1, see remarks (page 11), filed 07/22/2026, Applicant/s state/s “claim 9 has been amended to change each occurrence of "bus drum" to "bass drum," consistent with the specification (see, e.g., [0085] and [0201]-[0212] of the published application). Withdrawal of the objection is respectfully requested.”
Regarding Argument 2, see remarks (pages 11-12), filed 07/22/2026, Applicant/s state/s “Regarding the interpretation of certain claim limitations under 35 U.S.C. § 112(f), the claims have been amended so that they no longer recite the "first obtainer," "recommended information outputter," "second obtainer," or "answer information obtainer" limitations identified in the Office Action, and instead recite processing circuitry configured to perform the recited operations (see, e.g., [0054], [0068], [0531], and [0605]-[0606] of the published application, describing implementation by a CPU executing a program or by dedicated hardware such as an integrated circuit). Applicant respectfully submits that the amended claims recite sufficient structure and are not to be interpreted under 35 U.S.C. § 112(f).”
Regarding Argument 3, see remarks (pages 12-15), filed 07/22/2026, Applicant/s state/s “claims 1-20 are believed to be patent eligible, and withdrawal of the rejection under 35 U.S.C § 101 is respectfully requested.”
Regarding Argument 4, see remarks (page 15), filed 07/22/2026, Applicant/s state/s “Regarding the rejection of claims 1, 3, 10, 17, and 18 under 35 U.S.C § 102(a)(1) as anticipated by Yoshihama, withdrawal is respectfully requested. … The virtual drum set of Yoshihama is not a physical instrument; its layout is position coordinate data stored by the device itself (Yoshihama, [0014], [0035]-[0036]). Amended claim 1 requires the processing circuitry to obtain a captured image of a drumming instrument and a body of a performer and to extract, from the captured image, configuration information indicating a current physical setting of the drumming instrument and body information regarding the body of the performer. The portions of Yoshihama cited in the Office Action describe identifying the position coordinates of the markers of the stick units (Yoshihama, [0012]-[0014], [0019], [0023]-[0024]). Identifying positions of markers on drumsticks is not extracting, from a captured image, configuration information indicating a current physical setting of a drumming instrument, and is not extracting body information regarding the body of the performer. Nor does Yoshihama describe outputting the recited recommended information: the cited portions describe displaying a determined skill level (Yoshihama, [0061]) and enlarging a virtual pad region according to a degree of variation of detected shot positions (Yoshihama, [0058]-[0059]). Neither is recommended information indicating a recommended position, a recommended height, or a recommended inclination of a part of a drumming instrument.”
Regarding Argument 5, see remarks (page 16), filed 07/22/2026, Applicant/s state/s “Regarding claim 10, Yoshihama also does not describe outputting, as the recommended information, difference information indicating a difference between the configuration information and standard configuration information indicating a standard configuration for the drumming instrument. The Office Action relies on a right-hand region being set without change in position and size while a left-hand region is set with a larger size (Yoshihama, [0059]) and on display of the virtual plane (Yoshihama, [0036]). At most, these portions describe two regions having different sizes; they do not describe outputting difference information, and Yoshihama does not describe any standard configuration for a drumming instrument. Independent claims 17 and 18 recite features corresponding to those of claim 1 and are believed allowable for at least the same reasons. Accordingly, claims 1, 3, 10, 17, and 18 are believed to be novel over Yoshihama.”
Regarding Argument 6, see remarks (pages 16-17), filed 07/22/2026, Applicant/s state/s “The Office Action further rejects claims 2, 4, and 13-15 under 35 U.S.C. § 103 as unpatentable over Yoshihama in view of Furuya; claims 5, 8, 9, 11, and 12 over Yoshihama in view of Lee; claims 6 and 7 over Yoshihama in view of Lee and Furuya; and claim 16 over Yoshihama in view of Humphrey. Each of these claims depends directly or indirectly from claim 1 and includes all of the limitations of amended claim 1. As discussed above, Yoshihama does not describe at least extracting, from a captured image, configuration information indicating a current physical setting of a drumming instrument and body information regarding the body of the performer, or outputting recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument. The Office Action does not rely on Furuya, Lee, or Humphrey for these features, and the cited portions of those references do not cure the deficiency. The combinations therefore fail to teach or suggest all of the limitations of the claims as amended.”
Regarding Argument 7, see remarks (page 17), filed 07/22/2026, Applicant/s state/s “Moreover, regarding claim 13, the cited portions of Furuya describe outputting performance sensing data associated with two compared feature amounts superimposed and displayed (Furuya, [0194]-[0195]); superimposing items of sensing data is not displaying a virtual image of the body of the performer superimposed over a real image of the body of the performer.”
Regarding Argument 8, see remarks (page 17), filed 07/22/2026, Applicant/s state/s “Regarding claim 12, the cited portion of Lee describes displaying a symbol representing each drum element overlaid over captured pictures (Lee, [0193]); the Office Action does not identify any real image of a drumming instrument in Lee over which a virtual image of the drumming instrument is superimposed.”
Regarding Argument 9, see remarks (page 17), filed 07/22/2026, Applicant/s state/s “Regarding claim 16, the cited portions of Humphrey describe a help page from which a user can obtain information and an output module that transmits audio and video data to an output device (Humphrey, [0023], [0046], [0073], [0143]); none of these describes obtaining answer information indicating an answer to a questionnaire that is input from the performer, much less outputting recommended information based on such answer information. For at least these reasons, withdrawal of the rejections under 35 U.S.C. § 103 is respectfully requested.”
Examiner’s Responses:
In response to Argument 1, see remarks (page 11), filed 07/22/2026, with respect to the objection to claim 9 has been fully considered and is persuasive. The objection to claim 9 has been withdrawn.
In response to Argument 2, see remarks (pages 11-12), filed 07/22/2026, with respect to the interpretation of certain claim limitations under 35 U.S.C. § 112(f) have been fully considered and are persuasive. The interpretation of claims 1-2, 4-6, and 10-16 under 35 U.S.C. § 112(f) has been withdrawn.
In response to Argument 3, see remarks (pages 12-15), filed 07/22/2026, with respect to the 35 U.S.C § 101 rejection of claims 1-18 have been fully considered but they are not persuasive. Accordingly, the Examiner is maintaining the 35 U.S.C § 101 rejection of claims 1-18 and is rejecting newly added claims 19 and 20 under 35 U.S.C § 101.
Specifically, regarding the specific argument found on page 12 of Applicant’s remarks, filed 07/22/2026, “the extraction of such information from a captured image is performed, for example, by edge detection, semantic segmentation, frame (bone) estimation, and size calculation using an object of known size in the captured image as a basis for scaling”, the claims do not recite or suggest performing these steps, besides size calculation, which can also be practically performed in the human mind (i.e., a human can look at an object and determine its approximate size.). For example, Claim 1, as written, states “extract, from the captured image, configuration information…”. Claim 1 does not suggest performing specific image processing steps/methods (segmentation, edge detection, etc.) to extract this information. Instead, the Claim recites the data analysis steps recited at a high level of generality such that they could practically be performed in the human mind. See Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
Further, regarding the specific argument found on page 13 of Applicant’s remarks, filed 07/22/2026, which states “Under the USPTO memorandum of August 4, 2025, "Reminders on evaluating subject matter eligibility of claims under 35 U.S.C. 101," a claim does not recite a mental process when it contains limitations that cannot practically be performed in the human mind, and examiners are reminded not to expand the mental-process grouping to encompass such limitations. The rejection reasons that the recited information can be obtained, and the recommendation made, by a human such as a music teacher simply observing the drum set and the performer. That analogy does not account for the limitations of the amended claims: a human mind cannot extract configuration information or body information from a captured image by image processing, and Electric Power Group concerned the collection and analysis of information that humans can review, not pixel- level processing of captured images.” The Examiner states that the amended Claim 1 (and 17 and 18) limitations as well as the dependent claims do not state or suggest performing “pixel-level processing of captured images”. Instead, Claims 1, 17, and 18 limitations are recited at a high level of generality. Specifically, Claim 1 states “extract, from the captured image, configuration information…”, for example. The claim does not state the configuration information is extracted using any specific kind of image processing technique. “Extract(ing)” configuration information regarding a drum set up, as currently written in the claims, can be practically performed by the human mind,
Next, regarding Applicant’s statement, “See also USPTO Subject Matter Eligibility Example 39, in which a claim to training a neural network on collected and transformed digital facial images was determined not to recite a judicial exception, in part because the recited steps are not practically performed in the human mind.” The Examiner asserts that Applicant’s claims differ from those in USPTO Subject Matter Eligibility Example 39. Specifically, USPTO Subject Matter Eligibility Example 39 – Method for Training a Neural Network for Facial Detection does not recite a mental process because the steps are not practically performed in the human mind. For example, in USPTO Subject Matter Eligibility Example 39, the claim states “applying one or more transformations to each digital facial images including mirroring, rotating, smoothing, or contrast reduction to create a modified set of digital facial images” and “training a neural network…”. The instant application differs because the listed steps, as written, can be practically performed in the human mind. See the rejection below for further explanation.
Further, regarding Applicant’s assertion that “the claims as amended integrate any such exception into a practical application under Step 2A, Prong Two of the eligibility analysis (MPEP § 2106.04(d)). The August 4, 2025 memorandum cautions examiners not to oversimplify claim limitations or expand the application of the "apply it" consideration of MPEP § 2106.05(f), and instructs that the additional limitations be considered as an ordered combination rather than in a vacuum. Considered as an ordered combination, the claims recite a specific image-based measurement and recommendation technique that produces performer-specific, reproducible physical settings of a drumming instrument from a captured image, reducing setup time and enhancing reproducibility of the settings (see [0494]-[0504] of the published application).” Even considering the claims as an “ordered combination”, the Examiner asserts the Claim does not recite additional elements that amount to significantly more than the judicial exception. A music teacher can easily see the position/configuration of musical instruments (drums) and body position of a performer and provide recommendations regarding adjustments to both the instrument and the performer’s position. The Examiner finds that Claims 2-16 and 19-20 do not state significantly more since the claim only recites applying machine learning techniques to perform identifying/evaluating that is (as recited at a high level of generality) otherwise conventionally performed visually/mentally in a teaching/instruction environment. Even if assisting with drum set-up is useful/practical – the utility of the exception itself does not serve for integration into a practical application (see MPEP 2106.04(d)).
Further, regarding Applicant’s assertion on page 15 that “the present claims recite a specific image-processing pipeline that generates the configuration information and the body information and an output directed to a physical setting of a part of the drumming instrument. The dependent claims additionally recite augmented-reality display of virtual images superimposed over real images (claims 12 and 13) and size calculation using a scaling basis in the captured image (claim 19).” The Examiner states that superimposing and scaling objects in an image do not amount to significantly more than the judicial exception. Superimposing and scaling are well-understood, routine, and conventional activities previously known in the image processing industry.
In response to Argument 4, see remarks (pages 15-16), filed 07/22/2026, with respect to the 35 U.S.C § 102(a)(1) rejection of Claim 1 have been fully considered but they are not persuasive. See the Examiner’s specific responses/justifications regarding the 35 U.S.C § 102(a)(1) rejection of Claim 1 below.
Specifically, regarding Applicant’s argument that “The virtual drum set of Yoshihama is not a physical instrument; its layout is position coordinate data stored by the device itself”, the Examiner asserts that nowhere in Applicant’s claims does it state that the drumming instrument must be a “physical instrument”. Therefore, under Broadest Reasonable Interpretation (BRI), the Examiner interprets a virtual drum set to be a drumming instrument/drum set.
Next, regarding Applicant’s argument that “identifying positions of markers on drumsticks is not extracting, from a captured image, configuration information indicating a current physical setting of a drumming instrument, and is not extracting body information regarding the body of the performer” is not persuasive. Specifically, in light of paragraph [0162] of Applicant’s specification, which states, “It is to be noted that the drumsticks 34a and 34b are held by the performer 2. Therefore, information regarding the drumsticks 34a and 34b can be used as the body information. In contrast, the information regarding the drumsticks 34a and 34b may also be used as the configuration information regarding the configuration of the drum set 4.” The Examiner asserts that Yoshihama teaches the claim language “extract, from the captured image, configuration information regarding a configuration of the drumming instrument the configuration information indicating a current physical setting of the drumming instrument.” Yoshihama teaches determining the position coordinates (locations) of the marker unit (15) of the stick unit (10) (drumsticks 10R and 10L “drumming instrument”) within the captured image, determined by the image sensor unit (24) or by the CPU (21) (Paragraph [0030]). Yoshihama teaches the position coordinate data and marker feature information of the marker unit (15) of the stick unit (10) is calculated for the right and left marker units (Paragraphs [0028], [0046]). Further, Yoshihama teaches position coordinates, size, and angle of the marker portion (15) of the right-hand stick portion (10R) and the left-hand stick portion (10L) are detected by the image sensor unit (Paragraphs [0028], [0046]). The Examiner interprets position coordinates of the markers on the stick units (drumsticks) is “configuration information” since they provide information regarding how the drumsticks are being played or are configured, and their position in space. In addition, the Examiner interprets the position coordinate data for the marker units on the drumsticks is regarding a “current physical setting” since under BRI, the word setting can be interpreted as “location”. Therefore, calculating the position coordinates of the physical/tangible drumming instruments (drumsticks/ “stick units”) is “regarding” a current physical setting of the drumsticks i.e., “stick unit”. The Examiner also finds that Yoshihama reads on the claim language “extract, from the captured image, body information regarding the body of the performer playing the drumming instrument”. Yoshihama teaches capturing an image space with a camera unit (20) that includes the performer holding the stick (drumsticks) and performing a musical action. The camera unit distinguishes between right-(10R) and left-hand (10L) stick marker sections (Paragraphs [0015], [0019], [0023-24]). The Examiner interprets identifying information (position coordinates) regarding both the right-hand and left-hand drumsticks and their movement in space is under BRI, “body information regarding the body of the performer playing the drum instrument” since the position coordinates provide information regarding how the performer’s body is moving and how each hand (left and right) is moving in the imaging space.
Regarding Applicant’s assertion that “(nor does) Yoshihama describe outputting the recited recommended information; the cited portions describe displaying a determined skill level… and enlarging a virtual pad region according to the degree of variation of shot positions… Neither is recommended information indicating a recommended position, a recommended height, or a recommended inclination of a part of a drumming instrument.” The Examiner disagrees. Specifically, Yoshihama teaches showing visually the reported performance skill level determined by the CPU (Paragraphs [0064-65]). Under BRI, a determined skill level is interpreted as “recommended information”, i.e., the skill level is the recommended skill level based on the degree of variation of hits by the performer, etc. Further, Yoshihama teaches setting the virtual pad positions of the virtual plane based on the degree of variation calculated according to the skill level of the performer. For example, if the shot target for a virtual pad is larger than a predefined threshold, then the size of the virtual pad is increased (Paragraphs [0058-59], [0062]). Further, Yoshihama teaches displaying the virtual plane along with the virtual pad (81) placement in the display device (35) (Paragraph [0036]). The Examiner interprets displaying the virtual pad placement to be outputting the “recommended information” i.e., the recommended/updated virtual pad sizes/positions which is based on the skill level/degree of variation in shot position (Paragraphs [0057-58]). Further, the Examiner asserts that Yoshihama teahces the claim limitation, “the recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument.” Specifically, Yoshihama teaches the CPU sets the position and the size of the virtual pad on the virtual plane based on the degree of variation calculated. For example, if a shot target is the virtual pad (81) and the degree of variation for both hands is greater than a predefined threshold, the CPU may change the position on the virtual plane (Paragraphs [0058-59]). Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed recommended information types is required (i.e., recommended position, height, or inclination).
In response to Argument 5, see remarks (page 16), filed 07/22/2026, with respect to the rejection of claim 10 under 35 U.S.C. 103 has been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Furuya et al. (U.S. Patent Pub. No. 20220398937 A1, hereafter referred to as Furuya). The details of the rejection are found below.
In response to Argument 6, see remarks (pages 16-17), filed 07/22/2026, with respect to the argument regarding specific teachings of Yoshihama and references used to reject dependent claims, namely Furuya, Lee, and Humphrey, have been fully considered but they are not persuasive. Therefore, the 35 U.S.C. 102(a)(1) rejection of claims 1, 17, and 18 as being anticipated by Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) is maintained.
Specifically, regarding Applicant’s argument that “Yoshihama does not describe at least extracting, from a captured image, configuration information indicating a current physical setting of a drumming instrument and body information regarding the body of the performer, or outputting recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument”, the Examiner asserts that Yoshihama does teach each of the listed limitations. The Examiner asserts that Yoshihama teaches “extracting, from a captured image, configuration information indicating a current physical setting of a drumming instrument”. Yoshihama teaches determining position coordinates of the marker unit (15) of the stick unit (10) within the captured image by the image sensor unit (24) or by the CPU (21) (Paragraph [0030]. The Examiner interprets position coordinates of the marker unit indicate a current configuration (position) of the physical/tangible stick units (drumsticks) which are held by the performer and are used to play the virtual instrument. Further, since the stick units (drumsticks) are tangible/physical items, the positions of the marker units on the stick units indicate a current “physical” setting (i.e., location) of the drumming instrument (stick unit/drumsticks). Next, the Examiner asserts Yoshihama teaches the limitation “and body information regarding the body of the performer”. Yoshihama teaches the camera unit distinguishes between marker sections (15) of the right-hand stick section (10R) and the marker section (15) of the left-hand stick section and acquires position coordinates of each left- and right-hand stick unit individually (Paragraphs [0019], [0024]). The Examiner interprets position coordinates of the marker units in the left- and right-hands of the performer is “body information regarding the body of the performer” since the information is regarding the movement of the left and right hands of the performer (hands are body parts of the performer). Further, Yoshihama also teaches “outputting recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument”. Specifically, Yoshihama teaches setting the position and size of the virtual pad on the virtual plane based on the degree of variation calculated. For example, the CPU (31) may change the position on the virtual plane (of the virtual pad) (Paragraph [0058]). Further, the Examiner asserts that the recommended information (size and position of the virtual pad adjusted based on degree of variation/skill level of the performer) is “outputted” since the virtual plane along with the virtual pad (81) placement is displayed in the displayed device (35) (Paragraph [0036]). Lastly, regarding the portion of the argument which states, “The Office Action does not rely on Furuya, Lee, or Humphrey for these features, and the cited portions of those references do not cure the deficiency. The combinations therefore fail to teach or suggest all of the limitations of the claims as amended”, the Examiner asserts that Furuya, Lee, or Humphrey are not required to teach the amended features since Yoshihama teaches each of the claimed features in Claim 1.
In response to Argument 7, see remarks (page 17), filed 07/22/2026, with respect to the rejection of Claim 13 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Humphrey et al. (U.S. Patent Pub. No. 20120151344 A1, hereafter referred to as Humphrey). The details of the rejection are found below.
In response to Argument 8, see remarks (page 17), filed 07/22/2026, with respect to the rejection of Claim 12 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Asami et al. (U.S. Patent Pub. No. 20150141102 A1, hereafter referred to as Asami). The details of the rejection are found below.
In response to Argument 9, see remarks (page 17), filed 07/22/2026, with respect to the rejection of Claim 16 under 35 U.S.C. have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Furuya et al. (U.S. Patent Pub. No. 20220398937 A1, hereafter referred to as Furuya). The details of the rejection are found below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1 and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-20 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1, and similar rationale applies to independent Claims 17 and 18. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a machine since the claim is directed to an apparatus that allows for assisting with setting up a drum.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitations of:
obtain a captured image of a drumming instrument and a body of a performer playing the drumming instrument; extract, from the captured image, configuration information regarding a configuration of the drumming instrument, the configuration information indicating a current physical setting of the drumming instrument; extract, from the captured image, body information regarding the body of the performer; and output recommended information based on the configuration information and the body information extracted from the captured image, the recommended information being regarding a setting for playing the drumming instrument and corresponding to the performer the recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument are directed to an abstract idea. The claim is related to mental process by performing a process that is directed to "collecting information, analyzing it, and displaying certain results of the collection and analysis," where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016). If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that Claim 1 limitations do not provide additional elements or combination of additional elements to a practical application since the claim is adding the words of “applying it” with more instructions to implement an abstract idea on a computer. See MPEP 2106.05(f). As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do "‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’". Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. Specifically, the Examiner finds that the claim limitations of
“extract, … configuration information regarding a configuration of the drumming instrument” can be done by a human simply observing (looking at) a drum set to determine how the drum set is set up. Next, the Examiner finds that “extract(ing) … body information regarding the body of the performer” can be done by a human simply observing a person playing the drum set. For example, a music teacher/professional can observe a drum player’s posture, hand/leg placement, etc. Additionally, “output(ting) recommended information based on the configuration information and the body information” can be done mentally by a human, such as a music teacher or a professional who can observe the drum player with their eyes and make recommendations regarding the player’s posture and body position, as well as make recommendations regarding the height and/or position of various drum elements in the drum set corresponding to the performer.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claims merely recite a series of abstract ideas carried out by “an information processing apparatus.” Furthermore, the generic computer components of the processing circuitry, CRM, and processor recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
Claims 2-16 and 19-20 depending on the independent claims include all the limitation of the independent claim. The Examiner finds that Claims 2-16 and 19-20 do not state significantly more since the claim only recites applying machine learning techniques to perform identifying/evaluating that is (as recited at a high level of generality) otherwise conventionally performed visually/mentally in a teaching/instruction environment. For example, Claim 2 states “obtain the recommended information from a trained model by inputting at least one of the configuration information or the body information into the trained model, the trained model having been subjected to machine learning to estimate the recommended information.” Even if using machine learning model/models to assist with drum set-up is useful/practical – the utility of the exception itself does not serve for integration into a practical application (see MPEP 2106.04(d)).
Thus, Claims 2-16 and 19-20 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, the Examiner interprets that claims 1-20 are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3, 17, and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama).
Regarding Claim 1, Yoshihama teaches the following:
An information processing apparatus (Paragraph [0032], Fig. 5, Yoshihama teaches a center unit (30) which includes a CPU (31), ROM (32), RAM (33) and a switch control.)
PNG
media_image1.png
773
1062
media_image1.png
Greyscale
comprising:
processing circuitry (Paragraphs [0009], [0027-28], Fig. 4, Yoshihama teaches a hardware configuration of the camera unit (20) that makes up the performance device. The camera unit (20) includes a CPU (21), a ROM (22), a RAM (23), an image sensor unit (24) and a data communication unit (25). The CPU (21) controls the entire camera unit (20).)
PNG
media_image2.png
688
1029
media_image2.png
Greyscale
configured to:
obtain a captured image of a drumming instrument and a body of a performer playing the drumming instrument (Paragraphs [0012-13], [0030], Fig. 1, Fig. 3, Yoshihama teaches the camera unit (20) is configured as an optical imaging device and captures a space i.e., “imaging space” that includes the performer holding the stick (10) and performing a musical action, at a predetermined frame rate, and outputs it as moving image data. The stick (10) is a stick-shaped playing member that extends in the longitudinal direction. The performer holds one end of the stick (10) in their hand and performs playing actions such as swinging it up and down using their wrist as the pivot point. Under BRI, the Examiner interprets the stick unit (right-and left- hand stick units to be “drumming instrument(s)” since they are used to perform drum playing. Essentially, the “stick unit” is a set of drumsticks, which are used to play the virtual drum set. Drum sticks (the stick unit) are/is “drumming instruments”.)
[AltContent: arrow][AltContent: arrow]
PNG
media_image3.png
679
1032
media_image3.png
Greyscale
PNG
media_image4.png
131
257
media_image4.png
Greyscale
extract, from the captured image, configuration information (Paragraph [0030], Yoshihama teaches the position coordinates of the marker unit (15) of the stick unit (10) within the captured image may be determined by the image sensor unit (24) or by the CPU (21). The marker feature information of the captured marker unit (15) may also be determined by the image sensor unit (24).) regarding a configuration of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches the position coordinate data and marker feature information of the marker unit (15) detected by the image sensor unit (24), it calculates position coordinates of the marker units (15) (right marker and left marker) of the right-hand stick unit (10R) and the left-hand stick unit (10L). Under BRI, the Examiner interprets the stick unit (right-and left- hand stick units to be “drumming instrument(s)” since they are used to play the drums. The “stick unit” is a set of drumsticks, which are used to play the virtual drum set. Drumsticks (the stick unit) are/is “drumming instruments”.); and the configuration information indicating a current physical setting of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches position coordinates of the marker units (15). Position coordinates, size, and an angle of the marker portion (15) of the right-hand stick portion (10R) and the marker portion (15) of the left-hand stick portion (10L) are detected by the image sensor unit. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets the position coordinates of the marker portions of the sticks (drumsticks) to be a “physical setting” since under BRI, “setting” can mean a “manner, position, or direction in which something is set”. Therefore, the Examiner asserts that “position coordinates” of marker units i.e., drumsticks (tangible or “physical” item) indicate a “physical setting” of the drumsticks (drumming instrument).); extract, from the captured image, body information regarding the body of the performer playing the drumming instrument (Paragraphs [0019], [0024], [0030], Yoshihama teaches the camera unit (20) needs to distinguish between the marker section (15) of the right-hand stick section (10R) and the marker section (15) of the left-hand stick section (10L). The camera unit (20) can distinguish and acquire position coordinates of the marker unit (right marker) of the right-hand stick unit (10R) and the position coordinates of the marker unit (left marker) of the left-hand stick unit (10L) individually. The position coordinates of the marker unit (15) of the stick unit within the captured image may be determined by the sensor unit (24) or by the CPU. Under BRI, the Examiner interprets determining (extracting) the position coordinates of the marker unit (left- and right- hand stick sections) is extracting “body information” since it captures the location/positions of the marker sections in the left- and right-hands of the performer. Hands are body parts “regarding the body of the performer”. Position information of the marker sections provides information regarding the movement of the right- and left-hands of the performer.); and output recommended information (Paragraphs [0064-65], [0036], Yoshihama teaches the CPU (31) determines the performer’s skill level and notifies the determined skill level. Performers can assess their own performance skill level by looking at the reported performance skill level. It can be shown visually. In addition, CPU (31) displays the virtual plane along with the virtual pad (81) placement in the display device (35).) based on the configuration information (Paragraph [0046], Yoshihama teaches the CPU (21) executes the right marker detection process and left marker detection process. The CPU (21) acquires marker detection information such as the position coordinates, size, and angle of the marker portion (15) (right marker) of the right-hand stick portion (10R) and marker portion (15) (left marker) of the left-hand stick portion (10L) which are detected by the image sensor unit (24) and stored in the RAM.) and the body information extracted from the captured image (Paragraphs [0065], [0059], Yoshihama teaches the CPU (31) determines the performers skill level based on the distribution of the position coordinates of the right-hand stick (10R) and the distribution of the position coordinates of the left-hand stick (10L). The Examiner interprets distribution of position coordinates of the right- and left-hand sticks to be “body information” since it indicates the distribution of drum hits initiated by the performer’s right and left hands (body parts).), the recommended information being regarding a setting for playing the drumming instrument (Paragraphs [0058-59], [0062], [0036], Fig. 12, Yoshihama teaches setting the virtual pad positions of the virtual plane based on the degree of variation calculated and according to the skill of the performer. For example, if the shot target is virtual pad (81) and the degree of variation is larger than a predefined threshold, then the size of the virtual pad (81) is increased. The position of the virtual pad (81) may also be changed. Fig. 12 below shows an example of virtual pad settings where the drum pad (81) size has been increased. The CPU (31) displays the virtual plane along with virtual pad placement in the display device (35). The [AltContent: arrow]Examiner interprets “displaying” the virtual plane and virtual pad placement (81) by the display device is outputting recommended information (i.e., size and position of the virtual pads changed/adjusted based on the skill level of the performer).)
PNG
media_image5.png
507
635
media_image5.png
Greyscale
and corresponding to the performer (Paragraph [0056], Yoshihama teaches the CPU (31) performs a level determination. In this process, the CPU (31) determines the level indicating the performer’s skill level based on the degree of variation for both hands stored in RAM (33).) the recommended information indicating at least one of a recommended position, a recommended height, or a recommended inclination of a part of the drumming instrument (Paragraphs [0058-59], Yoshihama teaches the CPU (31) sets the position and size of the virtual pad on the virtual plane based on the degree of variation calculated. For example, if a shot target is the virtual pad (81) and the degree of variation for both hands is greater than a predefined threshold, the CPU may change the position on the virtual plane. Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed recommended information types is required (i.e., recommended position, height, or inclination).).
In regards to Claim 3, Yoshihama discloses the information processing apparatus according to claim 1, wherein the recommended information includes at least one of:
recommended configuration information regarding a recommended configuration of the drumming instrument for the body of the performer in playing the drumming instrument (Paragraphs [0058], [0062], Fig. 12, Yoshihama teaches increasing the size of virtual drum pad (81) as shown in Fig. 12. The position of the pad may also be changed. The arrangement of the virtual drum pads in the layout information can be changed according to the degree of variation when hitting the designated virtual pads, that is, according to the skill level of the performer. The Examiner interprets “at least one of” and “or” to mean only one of the limitations is required to meet the claim limitation.);
In regards to Claim 17, Yoshihama discloses:
A computer system-implemented method of processing information (Paragraph [0032], Yoshihama teaches a center unit (30) which is controlled by the CPU (31). CPU (31) executes programs stored in ROM (32).), the method comprising: obtaining a captured image of a drumming instrument and a body of a performer playing the drumming instrument (Paragraphs [0012-13], [0030], Fig. 1, Fig. 3, Yoshihama teaches the camera unit (20) is configured as an optical imaging device and captures a space “imaging space” that includes the performer holding the stick (10) and performing a musical action, at a predetermined frame rate, and outputs it as moving image data. The stick (10) is a stick-shaped playing member that extends in the longitudinal direction. The performer holds one end of the stick (10) in their hand and performs playing actions such as swinging it up and down using their wrist as the pivot point.); extracting, from the captured image, configuration information (Paragraph [0030], Yoshihama teaches the position coordinates of the marker unit (15) of the stick unit (10) within the captured image may be determined by the image sensor unit (24) or by the CPU (21). The marker feature information of the captured marker unit (15) may also be determined by the image sensor unit (24).) regarding a configuration of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches the position coordinate data and marker feature information of the marker unit (15) detected by the image sensor unit (24), it calculates position coordinates of the marker units (15) (right marker and left marker) of the right-hand stick unit (10R) and the left-hand stick unit (10L). Under BRI, the Examiner interprets the stick unit (right-and left- hand stick units to be “drumming instrument(s)” since they are used to perform drum playing. Essentially, the “stick unit” is a set of drumsticks, which are used to play the virtual drum set. Drum sticks (the stick unit) are/is “drumming instruments”.), the configuration information indicating a current physical setting of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches position coordinates of the marker units (15). Position coordinates, size, and an angle of the marker portion (15) of the right-hand stick portion (10R) and the marker portion (15) of the left-hand stick portion (10L) are detected by the image sensor unit. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets the position coordinates of the marker portions of the sticks (drumsticks) to be a “physical setting” since under BRI, “setting” can mean a “manner, position, or direction in which something is set”. Therefore, the Examiner asserts that “position coordinates” of drumsticks (tangible or “physical” item) indicate a “physical setting” of the drumsticks (drumming instrument).); extracting, from the captured image, body information regarding the body the performer (Paragraphs [0019], [0024], [0030], Yoshihama teaches the camera unit (20) needs to distinguish between the marker section (15) of the right-hand stick section (10R) and the marker section (15) of the left-hand stick section (10L). The camera unit (20) can distinguish and acquire position coordinates of the marker unit (right marker) of the right-hand stick unit (10R) and the position coordinates of the marker unit (left marker) of the left-hand stick unit (10L) individually. The position coordinates of the marker unit (15) of the stick unit within the captured image may be determined by the sensor unit (24) or by the CPU. Under BRI, the Examiner interprets determining (extracting) the position coordinates of the marker unit (left- and right- hand stick sections) is extracting “body information” since it captures the location/positions of the marker sections in the left- and right-hands of the performer. Hands are body parts “regarding the body of the performer”. Position information of the marker sections provides information regarding the movement of the right- and left-hands of the performer.); and outputting recommended information (Paragraphs [0064-65], [0036], Yoshihama teaches the CPU (31) determines the performer’s skill level and notifies the determined skill level. Performers can assess their own performance skill level by looking at the reported performance skill level. It can be shown visually. In addition, CPU (31) displays the virtual plane along with the virtual pad (81) placement in the display device (35).) based on the configuration information (Paragraph [0046], Yoshihama teaches the CPU (21) executes the right marker detection process and left marker detection process. The CPU (21) acquires marker detection information such as the position coordinates, size, and angle of the marker portion (15) (right marker) of the right-hand stick portion (10R) and marker portion (15) (left marker) of the left-hand stick portion (10L) which are detected by the image sensor unit (24) and stored in the RAM.) and the body information extracted from the captured image (Paragraphs [0065], [0059], Yoshihama teaches the CPU (31) determines the performers skill level based on the distribution of the position coordinates of the right-hand stick (10R) and the distribution of the position coordinates of the left-hand stick (10L). The Examiner interprets distribution of position coordinates of the right- and left-hand sticks to be “body information” since it indicates the distribution of drum hits initiated by the performer’s right and left hands (body parts).), the recommended information being regarding a setting for playing the drumming instrument (Paragraphs [0058-59], [0062], [0036], Fig. 12, Yoshihama teaches setting the virtual pad positions of the virtual plane based on the degree of variation calculated and according to the skill of the performer. For example, if the shot target is virtual pad (81) and the degree of variation is larger than a predefined threshold, then the size of the virtual pad (81) is increased. The position of the virtual pad (81) may also be changed. Fig. 12 below shows an example of virtual pad settings where the drum pad (81) size has been increased. The CPU (31) displays the virtual plane along with virtual pad placement in the display device (35). The Examiner interprets “displaying” the virtual plane and virtual pad placement (81) by the display device is outputting recommended information (i.e., size and position of the virtual pads based on the skill level of the performer).) and corresponding to the performer (Paragraph [0056], Yoshihama teaches the CPU (31) performs a level determination. In this process, the CPU (31) determines the level indicating the performer’s skill level based on the degree of variation for both hands stored in RAM (33).), the recommended information indicating at least one of a recommended position (Paragraphs [0058-59], Yoshihama teaches the CPU (31) sets the position and size of the virtual pad on the virtual plane based on the degree of variation calculated. For example, if a shot target is the virtual pad (81) and the degree of variation for both hands is greater than a predefined threshold, the CPU may change the position on the virtual plane. Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed recommended information types is required (i.e., recommended position, height, or inclination).),
In regards to Claim 18, Yoshihama discloses the following:
A non-transitory computer-readable storage medium storing a program which, when executed by at least one processor, causes the at least one processor to (Paragraph [0032], Fig. 5, Yoshihama teaches a hardware configuration of the center unit (30). The center unit includes a CPU (31) and ROM (32). ROM (32) stores processing programs for various processes executed by CPU (31).): obtain a captured image of a drumming instrument and a body of a performer playing the drumming instrument (Paragraphs [0012-13], [0030], Fig. 1, Fig. 3, Yoshihama teaches the camera unit (20) is configured as an optical imaging device and captures a space “imaging space” that includes the performer holding the stick (10) and performing a musical action, at a predetermined frame rate, and outputs it as moving image data. The stick (10) is a stick-shaped playing member that extends in the longitudinal direction. The performer holds one end of the stick (10) in their hand and performs playing actions such as swinging it up and down using their wrist as the pivot point.); extract, from the captured image, configuration information (Paragraph [0030], Yoshihama teaches the position coordinates of the marker unit (15) of the stick unit (10) within the captured image may be determined by the image sensor unit (24) or by the CPU (21). The marker feature information of the captured marker unit (15) may also be determined by the image sensor unit (24).) regarding a configuration of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches the position coordinate data and marker feature information of the marker unit (15) detected by the image sensor unit (24), it calculates position coordinates of the marker units (15) (right marker and left marker) of the right-hand stick unit (10R) and the left-hand stick unit (10L). Under BRI, the Examiner interprets the stick unit (right-and left- hand stick units to be “drumming instrument(s)” since they are used to perform drum playing. Essentially, the “stick unit” is a set of drumsticks, which are used to play the virtual drum set. Drum sticks (the stick unit) are/is “drumming instruments”.), the configuration information indicating a current physical setting of the drumming instrument (Paragraphs [0028], [0046], Yoshihama teaches position coordinates of the marker units (15). Position coordinates, size, and an angle of the marker portion (15) of the right-hand stick portion (10R) and the marker portion (15) of the left-hand stick portion (10L) are detected by the image sensor unit. Under Broadest Reasonable Interpretation (BRI), the Examiner interprets the position coordinates of the marker portions of the sticks (drumsticks) to be a “physical setting” since under BRI, “setting” can mean a “manner, position, or direction in which something is set”. Therefore, the Examiner asserts that “position coordinates” of drumsticks (tangible or “physical” item) indicate a “physical setting” of the drumsticks (drumming instrument).); extract, from the captured image, body information regarding the body of the performer playing the drumming instrument (Paragraphs [0019], [0024], [0030], Yoshihama teaches the camera unit (20) needs to distinguish between the marker section (15) of the right-hand stick section (10R) and the marker section (15) of the left-hand stick section (10L). The camera unit (20) can distinguish and acquire position coordinates of the marker unit (right marker) of the right-hand stick unit (10R) and the position coordinates of the marker unit (left marker) of the left-hand stick unit (10L) individually. The position coordinates of the marker unit (15) of the stick unit within the captured image may be determined by the sensor unit (24) or by the CPU. Under BRI, the Examiner interprets determining (extracting) the position coordinates of the marker unit (left- and right- hand stick sections) is extracting “body information” since it captures the location/positions of the marker sections in the left- and right-hands of the performer. Hands are body parts “regarding the body of the performer”. Position information of the marker sections provides information regarding the movement of the right- and left-hands of the performer.); and output recommended information (Paragraphs [0064-65], [0036], Yoshihama teaches the CPU (31) determines the performer’s skill level and notifies the determined skill level. Performers can assess their own performance skill level by looking at the reported performance skill level. It can be shown visually. In addition, CPU (31) displays the virtual plane along with the virtual pad (81) placement in the display device (35).) based on the configuration information (Paragraph [0046], Yoshihama teaches the CPU (21) executes the right marker detection process and left marker detection process. The CPU (21) acquires marker detection information such as the position coordinates, size, and angle of the marker portion (15) (right marker) of the right-hand stick portion (10R) and marker portion (15) (left marker) of the left-hand stick portion (10L) which are detected by the image sensor unit (24) and stored in the RAM.) and the body information (Paragraphs [0065], [0059], Yoshihama teaches the CPU (31) determines the performers skill level based on the distribution of the position coordinates of the right-hand stick (10R) and the distribution of the position coordinates of the left-hand stick (10L). The Examiner interprets distribution of position coordinates of the right- and left-hand sticks to be “body information” since it indicates the distribution of drum hits initiated by the performer’s right and left hands (body parts).), the recommended information being regarding a setting for playing the drumming instrument (Paragraphs [0058-59], [0062], [0036], Fig. 12, Yoshihama teaches setting the virtual pad positions of the virtual plane based on the degree of variation calculated and according to the skill of the performer. For example, if the shot target is virtual pad (81) and the degree of variation is larger than a predefined threshold, then the size of the virtual pad (81) is increased. The position of the virtual pad (81) may also be changed. Fig. 12 below shows an example of virtual pad settings where the drum pad (81) size has been increased. The CPU (31) displays the virtual plane along with virtual pad placement in the display device (35). The Examiner interprets “displaying” the virtual plane and virtual pad placement (81) by the display device is outputting recommended information (i.e., size and position of the virtual pads based on the skill level of the performer).) and corresponding to the performer (Paragraph [0056], Yoshihama teaches the CPU (31) performs a level determination. In this process, the CPU (31) determines the level indicating the performer’s skill level based on the degree of variation for both hands stored in RAM (33).), the recommended information indicating at least one of a recommended position (Paragraphs [0058-59], Yoshihama teaches the CPU (31) sets the position and size of the virtual pad on the virtual plane based on the degree of variation calculated. For example, if a shot target is the virtual pad (81) and the degree of variation for both hands is greater than a predefined threshold, the CPU may change the position on the virtual plane. Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed recommended information types is required (i.e., recommended position, height, or inclination).),
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4-5, 7-9, and 11 are rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Lee et al. (U.S. Patent Pub No. 20130118339 A1, hereafter referred to as Lee).
In regards to Claim 4, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is further configured to obtain at least one of: input configuration information input from the performer and regarding the configuration of the drumming instrument.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches wherein the processing circuitry is further configured to obtain at least one of: input configuration information input from the performer and regarding the configuration of the drumming instrument (Paragraphs [0045], [0179-181], Lee teaches a drum kit configuration phase in which the user chooses the components of their drum kit and guides them in placing those components within the space where they intend to drum. For example, the user may repeatedly pick their drum elements such as a tom drum, 19” ride cymbal, bass drum, etc. from lists.); Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed items is required to meet the claim limitation.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by having the user/performer input/choose the drum set components such as the tom drum, cymbal, bass drum, etc. that is taught by Lee, to make the invention that allows the user to personalize their drum set configuration/set-up; thus, one of ordinary skilled in the art would be motivated to combine the references since the user input allows the user to set up the drum set to match their playing style, musical genre they will be playing, personal preferences, and performance environment.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 5, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the configuration information includes information regarding a movable range of each of parts of the drumming instrument.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches wherein the configuration information includes information regarding a movable range of each of parts of the drumming instrument (Paragraph [0157-159], [0162], [0194], Lee teaches computing the “openness” or “range” (“hh_range”) of the hi-hat cymbal by examining the y-coordinate of the hi-hat foot marker. hh_range is set to the maximum y-coordinate value minus the minimum y-coordinate value. The Examiner interprets determining the openness of the hi-hat cymbal is a moveable range of a part of the drumming instrument.), and
the processing circuitry (Claim 20, Lee teaches causing a processor to perform a method.) is configured to output the recommended information based on the information regarding the movable range (Paragraphs [0193-194], Lee teaches if a hi-hat cymbal is present in the drum kit composition, the computer program instructs the user to open and close the hi-hat with the relevant foot. The computer program records the minimum and maximum y coordinates of the corresponding foot marker over the arming and swinging phases. “hh_range” is set to the maximum value minus the minimum value. A symbol representing each drum element is displayed overlaid over the captured pictures at its position.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by capturing information regarding the movable range of the hi-hat cymbal that is taught by Lee, to make the invention that identifies the movable range for the drum elements which are movable; thus, one of ordinary skilled in the art would be motivated to combine the references since determining the level of “openness” for when the hi-hat cymbal is hit by the performer allows for the computer system to play the proper corresponding sound for the level of “openness” for the hi-hat cymbal detected by the imager (Lee, Paragraph [0161]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 7, Yoshihama discloses the information processing apparatus according to claim 3, wherein the recommended configuration information includes at least one of: a position of each of parts of the drumming instrument (Paragraph [0036], Fig. 7, Yoshihama teaches arranging virtual drum pads (81, 82, 83, and 84) based on position data.); (Paragraph [0041], Fig. 8, Yoshihama teaches a stick part processing, which includes performing attitude detection processing of the drumsticks (“stick unit”) based on the motion sensor information. Attitude of the drumsticks (“stick unit”) are/is calculated, such as the roll angle and pitch angle of the drumsticks (“stick unit”). Under Broadest Reasonable Interpretation, the Examiner interprets the drumsticks to be “drumming instruments” since the claim is silent to the definition of drumming instrument.);
PNG
media_image6.png
672
648
media_image6.png
Greyscale
or a positional relationship between the parts of the drumming instrument (Paragraphs [0035-36], Fig.7, Yoshihama teaches storing layout information containing information for n drum pads, from first pad to nth pad. Each pad’s information is associated with and stored with the pad, such as its position (position coordinates in virtual plane) and its size (shape and diameter).), and
Yoshihama does not explicitly disclose a height of the each part of the drumming instrument and the recommended body information includes at least one of: a position of each of parts of the body of the performer; a height of the each part of the body of the performer; a posture of the each part of the body of the performer; or a positional relationship between the parts of the body of the performer.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches a height of the each part of the drumming instrument (Paragraphs [0192-193], Lee teaches for each drum element, setting the coordinates (x_d, y_d) of the element and displaying a symbol representing each drum element over the captured pictures at its position (x_d, y_d). The Examiner interprets the y-coordinate to be the “height” as shown in Fig. 7 (704).) and the recommended body information includes at least one of: or a positional relationship between the parts of the body of the performer (Paragraphs [0182-191], Lee teaches a computer program which instructs the user to remain still in a natural drumming posture for two seconds. In this posture, the user should hold their hands and/or drumsticks so the markers are equidistant from their torso, below their neck, and above their waist. The user should not cross their arms, wrists or drumsticks. The computer program continuously checks for the presence of 4 markers by computing the distance formula.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by providing a recommended (pre-defined) height for each drum element and a recommended drumming position that is taught by Lee, to make the invention that enables the user to play the drums right away, provides the user with a realistic drumming experience and ensures the user maintains a proper drumming form while they play the drums; thus, one of ordinary skilled in the art would be motivated to combine the references since in a proper drumming posture, the user should hold their hands and/or drumsticks so they are equidistant from their torso, below their neck, above their waist, an without their arms/wrists/hands crossed and proper marker identification is required in determining which sound to play when the user hits a specific drum element (Lee, Paragraph [0187]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 8, Yoshihama discloses the information processing apparatus according to claim 3, wherein the drumming instrument is a drum set (Paragraph [0067], Fig. 1(2), Yoshihama teaches a virtual drum set D.),
PNG
media_image7.png
513
732
media_image7.png
Greyscale
Yoshihama does not explicitly disclose:
and the recommended configuration information includes information regarding at least one of:
a position and a height of a chair;
a position and an inclination of a bass drum;
a position, a height, and an inclination of a snare drum;
a position and a height of a hi-hat cymbal;
a position, a height, and an inclination of a floor tom;
a position, a height, and an inclination of a tom;
a position, a height, and an inclination of a cymbal;
or a difference between the height of the snare drum and the height of the hi-hat cymbal.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches the recommended configuration information includes information regarding at least one of:
a position and a height of a hi-hat cymbal (Paragraphs [0157-166], [0192-193], Lee teaches determining the openness of the hi-hat cymbal drum element. For each drum element, the coordinates (x_d, y_d) and size (s_d) are set to pre-defined values. A symbol representing each drum element is displayed overlaid over the captured pictures at its position (x_d, y_d). The computer program records the minimum and maximum y-coordinates (702) of the corresponding marker. hh_min is set to the minimum value and hh_range is set to the maximum value minus the minimum value. Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed items is required to meet the claim limitation.);
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by setting the pre-defined coordinates and size of the hi-hat cymbal and the minimum and maximum y-coordinates (max y-coordinate is height) and displaying the position of the drum element over the captured pictures that is taught by Lee, to make the invention that provides the user with a recommended pre-defined drum set-up configuration rather than having to manually create the drum kit by repeatedly picking drum elements; thus, one of ordinary skilled in the art would be motivated to combine the references since setting the drum set to pre-defined values enables the user to start playing right away, which saves the player time (Lee, Paragraph [0192]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 9, Yoshihama discloses the information processing apparatus according to claim 3, wherein the drumming instrument is a drum set (Paragraph [0067], Fig. 1(2), Yoshihama teaches a virtual drum set D.).
Yoshihama does not explicitly disclose the recommended body information includes information regarding at least one of:
an angle of a thigh of the performer sitting on a chair;
a positional relationship between a bass drum and a leg of the performer playing the bass drum;
an angle of a shin of the leg of the performer playing the bass drum;
a positional relationship between a snare drum and the body of the performer;
a positional relationship between an upper arm and the body of the performer playing the snare drum;
a relationship between a height of the snare drum,
a height of an elbow of the performer,
and a height of a knee of the performer;
a positional relationship between a hi-hat cymbal and the leg of the performer playing the hi-hat cymbal;
the angle of the shin of the leg of the performer playing the hi-hat cymbal;
a positional relationship between a floor tom and the body of the performer;
a positional relationship between a tom and an arm of the performer;
or a positional relationship between a cymbal and the arm of the performer.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches the recommended body information includes information regarding at least one of:
a positional relationship between a tom and an arm of the performer (Paragraphs [0120], [0138], Lee teaches determining what drum kit element is deemed reachable by a specific hand of the performer. For example, the drumstick held in the left hand is deemed to be usable to hit all drum elements except for the floor tom. The Examiner interprets that the performers hand is attached to their arm and therefore establishes a positional relationship between the arm and the tom. Under BRI, the Examiner interprets “at least one of” and “or” to mean only one of the listed items is required to meet the claim limitation.);
or a positional relationship between a cymbal and the arm of the performer (Paragraph [0138], Lee teaches using a large expected marker size for a cymbal and placing the pre-defined cymbal position in front of the user, in line with the camera. The marker size is large if the hit occurs far from the user (i.e., arm is extended).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by determining a positional relationship between the performer’s body part (arm) and different drum elements such as the cymbal and tom that is taught by Lee, to make the invention that is able to distinguish between the user’s drum hits when there are two (or more) types of drums placed in front of the user and in line with the camera; thus, one of ordinary skilled in the art would be motivated to combine the references since understanding the positional relationship between the users arm and the different types of drums (tom and cymbal) enables the user to express their intention to hit one drum or the other even if their pre-defined positions within the picture appear to be identical (Lee, Paragraph [0138]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 11, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is configured to output the recommended information based on a motion of arranging parts of the drumming instrument in order.
Lee is in the same field of art of performing drum kit configuration and adjustment using imaging. Further, Lee teaches wherein the processing circuitry (Paragraph [0029], Lee teaches a computer program product operable to program a processor to perform any method described herein.) is configured to output the recommended information based on a motion of arranging parts of the drumming instrument in order (Paragraph [0192], Lee teaches a computer program which instructs the user to place the drum elements by making drumming gestures at the desired locations. Elements are played one at a time, by making a drumming gesture for each element after the computer has displayed the name of the element to place next. For each drum element, the coordinates and size of the drum are set.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by instructing the user to arrange the different drum elements one at a time in their desired locations that is taught by Lee, to make the invention that enables the user to place the drum elements in their preferred positions; thus, one of ordinary skilled in the art would be motivated to combine the references since the user should choose where the components of their drum kit will be placed to their liking and be guided in placing the components within the space they intend to drum (Lee, Paragraph [0045]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 2, 10, and 14-16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Furuya et al. (U.S. Patent Pub. No. 20220398937 A1, hereafter referred to as Furuya).
Regarding Claim 2, Yoshihama discloses the information processing apparatus according to claim 1, wherein processing circuitry (Paragraph [0032], Fig. 5, Yoshihama teaches a center unit (30) which includes a CPU (31), ROM (32), RAM (33) and a switch control.) is configured to obtain the recommended information (Paragraphs [0064-65], Yoshihama teaches the CPU (31) determines the performer’s skill level and notifies the determined skill level. Performers can assess their own performance skill level by looking at the reported performance skill level. It can be shown visually.)
Yoshihama does not explicitly disclose
Fuyura is in the same field of art of generating tailored feedback information for a performer playing a musical instrument. Further, Fuyura teaches obtain the recommended information from a trained model (Paragraph [0006], Furuya teaches presenting feedback information to the second user (trainee) based on the relationship information obtained from performing machine learning.) by inputting at least one of the (Paragraphs [0006], [0107-109], Fuyura teaches acquiring subjective evaluation information from a second user (trainee) about each performance performed by at least part of a body of a first user (role model) moving, and performing machine learning on a relationship between each performance and the corresponding subjective evaluation information. The trainee receives an output of the difference (deviation) between the measurement result (role model) of the motion element and the measurement result of the motion element during the performance of the trainee. Motion elements may include behaviors of joints, muscles, fingers, arms, etc. Under BRI, the Examiner interprets “at least one or” and “or” to mean only one of either the configuration information or the body information is required to meet the claim limitation.), the trained model having been subjected to machine learning to estimate the recommended information (Paragraphs [0184], [0187-188], Figs. 7 & 9, Furuya teaches performing machine learning according to a predetermined rule. The learning unit (346) stores relationship information and the like obtained by the machine learning in the storage unit (370), thereby constructing the database (372).).
Therefore, it would have been obvious to one having ordinary skill in the art before the
effective filing date of the claimed invention to modify the invention of Yoshihama by inputting the body movement information into a trained model to generate difference (deviation) information between the trainee and a "role model" that is taught by Furuya, to make the invention that outputs a deviation or a specific content to be corrected (body part, motion amount) to the trainee in order to make the performance of the trainee similar to the performance of the role model; thus, one of ordinary skilled in the art would be motivated to combine the references since providing feedback to the trainee in the form of a difference between the measurement result of the motion element of the trainee and the measurement result of the motion element of the role model enables the trainee to recognize the difference between the motion element in the performance element of the trainee and the role model, and consciously perform the performance so as to approach the performance of the role model, thereby improving the trainee (Furuya, Paragraph [0104]).
Thus, the claimed subject matter would have been obvious to a person having ordinary
skill in the art before the effective filing date of the claimed invention.
In regards to Claim 10, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is configured to output, as the recommended information, difference information indicating a difference between the configuration information and standard configuration information.
Furuya is in the same field of art of generating tailored feedback information for a performer playing a musical instrument. Further, Furuya teaches wherein the processing circuitry (Paragraph [0325], Furuya teaches the CPU (950) includes, for example, one or more processors including an arithmetic circuit such as a CPU, various processing circuits, and the like, and functions as the main control unit 340 that controls the entire information processing device 900.) is configured to output, as the recommended information, difference information (Paragraph [0111], Furuya teaches in order to make the performance of the trainee similar to the performance as a model, a motion element that is considered to have a high necessity of correction is subject to a comparison, and a deviation (difference) or a specific content to be corrected (a body part, a motion amount, or the like) is fed back to the trainee.) indicating a difference between the configuration information and standard configuration information (Paragraphs [0111], [0194-195], Furuya teaches comparing the feature amount of the collected sensing data with the feature amount of the sensing data as a model for the trainee and perform feedback to the trainee based on the comparison result. Under BRI, the Examiner interprets the trainee’s sensing data (such as a motion amount, a body part) to be “configuration information” and the model’s sensing data to be the “standard configuration information”.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by comparing the feature amount of the trainee with a feature amount (second feature amount) of a role model and calculates a difference/deviation between the role model and the trainee feature amounts that is taught by Furuya, to make the invention that generates feedback to the trainee using the difference information; thus, one of ordinary skilled in the art would be motivated to combine the references since identifying specific differences to be corrected (a body part, a motion amount, or the like) can be fed back to the trainee, allowing tailor-made feedback to be generated and fed to the trainee in order to improve their performance by addressing the differences (Furuya, Paragraph [0111]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 14, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is further configured to obtain information regarding a desired setting desired by the performer and carried out by the performer and output the recommended information based on the information regarding the desired setting.
Fuyura is in the same field of art of generating tailored feedback information for a performer playing a musical instrument. Further, Fuyura teaches wherein the processing circuitry (Paragraph [0325], Furuya teaches the CPU (950) includes, for example, one or more processors including an arithmetic circuit such as a CPU, various processing circuits, and the like.) is further configured to obtain information regarding a desired setting desired by the performer and carried out by the performer (Paragraphs [0178], [0194], Furuya teaches the trainee may input in advance of the subjective evaluation information, a favorite performance or a past performance of a player who the trainee desires to regard as a role model. The selected role model will be used for comparison to generate feedback to the trainee/player.) and output the recommended information based on the information regarding the desired setting (Paragraph [0194], Furuya teaches a comparison unit (350) which calculates a difference (deviation) between the selected role model and the calculated feature amount of the player. The comparison unit extracts information (body part, motion amount, etc.) about the motion element necessary for setting the calculated difference to zero. The comparison unit generates feedback information for feeding back to the trainee using the comparison result to output the generated feedback information to an output control unit.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by obtaining as input from the player, information regarding a desired role model whom they would like to be compared to and outputting feedback to the player regarding the deviations between the motions of the player/trainee and the role model that is taught by Furuya, to make the invention that takes as input, the performer's desired role model and outputs feedback regarding the differences between the player/trainee and the role model; thus, one of ordinary skilled in the art would be motivated to combine the references since feedback enables the trainee to recognize the difference between the motion element, such as behaviors of joints, muscles, fingers, arms, etc. in the performance of the trainee and the role model and consciously perform the performance so as to approach the performance of the model, thereby improving the performance of the trainee. (Furuya, Paragraphs [0104], [0109]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 15, Yoshihama in view of Fuyura discloses the information processing apparatus according to claim 14, wherein the processing circuitry (Paragraph [0325], Furuya teaches the CPU (950) includes, for example, one or more processors including an arithmetic circuit such as a CPU, various processing circuits, and the like.) is configured to output the recommended information using a trained model that has undergone machine learning using the information regarding the desired setting as teaching data (Paragraphs [0183], [0186-188], [0110], Furuya teaches a supervised learning unit (learning device) (346) such as a deep neural network. The learning unit stores the relationship information and the like obtained by the machine learning in the storage unit, thereby constructing the database (DB) (372). The collected sensing data, motion element characterizing the performance state, and coordination or non-coordination of the motion element can be stored in the database.).
Regarding Claim 16, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is further configured to obtain answer information indicating an answer to a questionnaire that is about playing of the drumming instrument and that is input from the performer, and output the recommended information based on the answer information.
Furuya is in the same field of art of generating tailored feedback information for a performer playing a musical instrument. Further, Furuya teaches wherein the processing circuitry is further configured to obtain answer information indicating an answer to a questionnaire that is about playing of the drumming instrument and that is input from the performer (Paragraphs [0167], [0175-176], Fig. 5, Furuya teaches an input unit (300) receives an input of data and a command to the server. The input unit (300) is realized by a touch panel, a keyboard, or the like, and can receive an input of attribute information of the trainee or the player. For example, as illustrated in Fig. 5, the trainee may be caused to designate the input of the subjective information with a combination of degrees of predetermined subjective evaluation indexes as a value of the subjective evaluation information via the presentation device (70). In the example illustrated in FIG. 5, first, the trainee is caused to input subjective evaluations of the past performance such as “IMPOSSIBLE”, “AMUSING”, and “POOR”. Furthermore, in this example, the trainee is caused to input three index values indicating the degree of closeness to each of “LIKE AND DISLIKE”, “FUN AND SAD”, and “ANGER AND FEAR” in each of the subjective evaluations of the past performances such as “IMPOSSIBLE”, “AMUSING”, and “POOR”.),
PNG
media_image8.png
426
664
media_image8.png
Greyscale
and output the recommended information based on the answer information (Paragraph [0173], Furuya teaches the data acquisition unit (342) acquires the subjective evaluation information from the trainee about each performance of the past player transmitted from the presentation device (70) to be described later to output the acquired subjective evaluation information to the processing unit.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by allowing input by the performer of self-evaluations of previous performances that is taught by Furuya, to make the invention that acquires subjective evaluation information from the performer to determine how the performer felt about the performance; thus, one of ordinary skilled in the art would be motivated to combine the references since gathering information regarding the performer’s opinion on the performance may allow the system to provided targeted feedback based on those inputs to improve the performer’s future performances.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Lee et al. (U.S. Patent Pub No. 20130118339 A1, hereafter referred to as Lee) in further view of Furuya et al. (U.S. Patent Pub. No. 20220398937 A1, hereafter referred to as Furuya).
Regarding Claim 6, Yoshihama in view of Lee discloses the information processing apparatus according to claim 5, wherein the processing circuitry (Paragraph [0032], Fig. 5, Yoshihama teaches a center unit (30) which includes a CPU (31), ROM (32), RAM (33) and a switch control.) is configured to output the recommended information (Paragraphs [0064-65], Yoshihama teaches the CPU (31) determines the performer’s skill level and notifies the determined skill level. Performers can assess their own performance skill level by looking at the reported performance skill level. It can be shown visually.)
Yoshihama in view of Lee does not explicitly disclose
Fuyura is in the same field of art of generating tailored feedback information for a performer playing a musical instrument. Further, Fuyura teaches wherein the processing circuitry is configured to output the recommended information using a trained model that has undergone machine learning (Paragraphs [0184], [0187-188], Figs. 7 & 9, Furuya teaches performing machine learning according to a predetermined rule. The learning unit (346) stores relationship information and the like obtained by the machine learning in the storage unit (370), thereby constructing the database (372).) using the information regarding the movable range as teaching data (Paragraphs [0145], [0173], [0290], Furuya teaches a position sensor for detecting the vertical movement of a keyboard by the motion element of the trainee of the player. For example, by installing a position sensor below each keyboard, it is possible to detect the vertical movement of the keyboard and detect the movement of the fingers that move the keyboard up and down. Sensing data is stored in a database.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama in view of Lee by using as teaching data, information regarding a movable range of an instrument element that is taught by Furuya, to make the invention that computes and outputs the difference between the trainee and role model's movements, such as a difference in vertical movement of a keyboard (or any instrument, such as drums); thus, one of ordinary skilled in the art would be motivated to combine the references since feeding back a difference (deviation) between the measurement result (role model) of the motion element (keyboard stroke, etc.) during the past performance and the measurement result of the motion element during the trainee's performance to effectively learn the expression desired by the trainee (Furuya, Paragraph [0108]). Additionally, the present embodiment can be applied to other instruments such as percussion instruments, such as drums (Furuya, Paragraph [0319]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Asami et al. (U.S. Patent Pub. No. 20150141102 A1, hereafter referred to as Asami).
Regarding Claim 12, Yoshihama teaches the information processing apparatus according to claim 3.
Yoshihama does not explicitly disclose wherein the processing circuitry is configured to display, as the recommended configuration information a virtual image of the drumming instrument superimposed over a real image of the drumming instrument.
Asami is in the same field of art of drumming instruments and providing recommended information (predetermined playing actions) to the performer. Further, Asami teaches wherein the processing circuitry is configured to display (Paragraph [0028], Asami teaches a game screen (35) is displayed upon monitor (4).), as the recommended configuration information a virtual image of the drumming instrument superimposed (Paragraphs [0029-30], Asami teaches the AR image (37) is an image in a virtual space, and is disposed above the actual space image (36) so as to extend into the actual space. The AR image (37) functions as an object in the extended space that is created by merging together the actual space and the virtual space. The AR image includes four drum images (DI) which serve as standard reference marks and an object image that serves as a command mark. The object images (OI) disappear from the game screen (35) after they have become superimposed upon the upper surfaces of the corresponding drum images (DI).)
PNG
media_image9.png
408
489
media_image9.png
Greyscale
over a real image of the drumming instrument (Paragraph [0028], Asami teaches as the actual imaging space (36), there is used an image which is photographed by the optical sensor (OS). The player who is performing playing actions and the four play operation units (8) are included in the range of photography of the optical sensor (OS). Due to this, a player image (PI) that corresponds to the player is included in the actual image space (36).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by superimposing the drum-shaped object images over the actual imaging space photographed by the optical sensor that is taught by Asami, to make the invention that superimposes the objects over the drum at the appropriate timings in the order at which the appropriate operations must be performed; thus, one of ordinary skilled in the art would be motivated to combine the references to indicate to the performer when to strike each drum pad, therefore expanding the range or play actions demanded of the player (Asami, Abstract).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Humphrey et al. (U.S. Patent Pub. No. 20120151344 A1, hereafter referred to as Humphrey).
Regarding Claim 13, Yoshihama discloses the information processing apparatus according to claim 3.
Yoshihama does not explicitly disclose wherein the processing circuitry is configured to display, as the recommended body information, a virtual image of the body of the performer superimposed over a real image of the body of the performer.
Humphrey is in the same field of art of providing visual recommendations to performing artists/musicians such as how to position their body while playing a musical instrument. Further, Humphrey teaches wherein the processing circuitry (Paragraph [0083], Humphrey teaches the system (200) comprising processor (205). The system also includes output device (275) to provide audio data and visual data to the user.) is configured to display, as the recommended body information, a virtual image of the body of the performer superimposed over a real image of the body of the performer (Paragraph [0122], Humphrey teaches a video camera feature (861) that allows users to video themselves playing and superimpose or compare the user captured video to instructional video. In addition, the Examiner interprets a video to encompass capturing an image in light of paragraph [0063] of Applicant’s specification, which states “the term “image” is intended to encompass a still image, a sequence of still images, multiple still images spaced throughout time, or images in the form of a video.”).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by superimposing the video of the performer playing and an instructional video that is taught by Humphrey, to make the invention that allows the performer to visually compare their performance to the performance of the instructor in the instructor video, thus, one of ordinary skilled in the art would be motivated to combine the references since a solution is needed in the art for learning how to emulate subtleties, such as gestures, body movements, and hand positions, ad then identifying the desired portion of a performance having the subtlety, as well as accurately and precisely selecting and practicing the desired portion until it is learned. By superimposing the performances, the performer may be able to identify and view the subtleties and help them in developing or honing in their skills (Humphrey, Paragraph [0007]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Tabata (U.S. Patent Pub. No. 20130239780 A1, hereafter referred to as Tabata).
Regarding Claim 19, Yoshihama discloses the information processing apparatus according to claim 1.
Yoshihama does not explicitly disclose wherein the processing circuitry is configured to calculate a size of at least one of a part of the drumming instrument or a part of the body of the performer using, as a basis for scaling, a size of an object that is included in the captured image and whose size is already known.
Tabata is in the same field of art of playing a virtual musical instrument and changing the size of the drum pads. Further, Tabata teaches wherein the processing circuitry (Paragraph [0106], Tabata teaches the CPU (31) teaches an enlargement/reduction rate used to scale (enlarge or reduce) the respective standard sizes of the plurality of virtual pads in the standard set layout information to sizes in the changed set layout information.) is configured to calculate a size of at least one of a part of the drumming instrument
(Paragraph [0105], Tabata teaches the CPU (31) calculates the center coordinates of the square from the respective position coordinates of the marker sections of the drumstick sections detected when the CPU (31) has judged that an operation to change the current set layout has been performed. The CPU (31) calculates the offset value between the center coordinates of the square in the drumstick standard position and the center coordinates of the square in the drumstick changed position. The Examiner interprets the “square” effectively calculates the coordinates (height) of the drumsticks.)
(Paragraphs [0106-108], Tabata teaches the CPU (31) multiplies all position coordinates included in areas defined by respective standard positions and standard sizes by the plurality of virtual pads (81) in the standard set layout information with the enlargement/reduction rates in the vertical and lateral directions, and adds the offset value to all position coordinates after the multiplication.) (The Examiner interprets “or” to mean only one of either the size of the drumming instrument or the body part of the performer is required to meet the claim limitation.) using, as a basis for scaling (Paragraphs [0106-108], Tabata teaches the enlargement/reduction rate is a scale used to enlarge or reduce the respective standard sizes of the plurality of virtual pads (81) in the standard set layout information to sizes in the changed set layout information. The CPU multiplies all position coordinates included in areas defined by the respective standard positions and standard sizes of the plurality of virtual pads (81) in the standard layout information with the enlargement/reduction rates in the vertical and lateral directions.), a size of an object (Paragraphs [0105-107], Tabata teaches the CPU (31) calculates the center coordinates of the square from the respective position coordinates of the marker sections of the drumstick sections detected when the CPU (31) has judged that an operation to change the current set layout has been performed. The CPU (31) calculates the offset value between the center coordinates of the square in the drumstick standard position and the center coordinates of the square in the drumstick changed position. The CPU (31) calculates the enlargement/reduction rate in the lateral direction and the enlargement reduction rate in the vertical direction. The Examiner interprets the “square” effectively calculates the coordinates (height, vertical direction) of the drumsticks.) that is included in the captured image (Paragraph [0057], Tabata teaches the identification of the position coordinates of the marker section (15) of the drumstick section (10) within a captured image may be performed by the image sensor section (24), or it may be captured by the CPU (21).) and whose size is already known (Paragraphs [0068], [0114], Tabata teaches “standard set layout information”, and a position and a size included in the standard layout information i.e., “standard size”. The standard set layout information includes standard positions and standard sizes that serve as reference for the arrangement of the virtual pads (81).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama by using the standard size of the virtual drum pad that is taught by Tabata, to make the invention that enlarges/reduces the size of the virtual drum pads based on the movement of the performer; thus, one of ordinary skilled in the art would be motivated to combine the references since
the arrangement of the plurality of virtual pads (81) is changed in accordance with the position of the instrument player. As a result, the instrument player need not play the instrument in an uncomfortable position (Tabata, Paragraphs [0113-115]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yoshihama (Japanese Patent Pub. No. JP2013-195582A, hereafter referred to as Yoshihama) in view of Furuya et al. (U.S. Patent Pub. No. 20220398937 A1, hereafter referred to as Furuya) in further view of Kohli et al. (U.S. Patent Pub. No. 20130156298 A1, hereafter referred to as Kohli).
Regarding Claim 20, Yoshihama in view of Furuya discloses the information processing apparatus according to claim 2.
Yoshihama in view of Furuya does not explicitly disclose wherein the trained model is one of a plurality of trained models each corresponding to a classification of a height of the performer, and the processing circuitry is configured to select, from among the plurality of trained models, the trained model corresponding to the height of the performer indicated by the body information.
Kohli is in the same field of art of using attributes such as height, body shape, gender, to guide processing of images for various tasks/to generate an output. Further, Kohli discloses wherein the trained model is one of a plurality of trained models each corresponding to a classification of a height of the performer (Paragraphs [0005], [0029], Abstract, Kohli teaches one or more random decision forests are trained using images where global variable values such as player height are known in addition to ground-truth data. Each random decision forest is trained using training data examples having global variable values which fall within one of the bins.), and the processing circuitry (Paragraph [0071], Kohli teaches computing-based device 1101 comprises one or more processors (1100) which may be microprocessors, controllers, or any type of processors for processing computer executable instructions to control the operation of the device in order to process images.) is configured to select, from among the plurality of trained models (Paragraph [0029], Fig. 2, Kohli teaches the global variable is discretized by dividing a range of potential global variable values into bins. A separate random forest may be trained for each global variable bin as illustrated in Fig. 2. Four random decision forests 204, 206, 208, 210 are shown. For example, in the case of player height as a global variable, four bins covering a total range from 0.5 m to 2 m may be used.), the trained model corresponding to the height of the performer (Paragraph [0029], Kohli teaches a separate random decision forest may be trained for each global variable bin. In the case of player height as the global variable, four bins may cover a range from 0.5 m to 2 m.)
PNG
media_image10.png
415
488
media_image10.png
Greyscale
indicated by the body information (Paragraphs [0030-31], [0027], Fig. 2, Kohli teaches when a test image (200) is received (an image not used for training) then global variable knowledge (202) for that test image is used to select which of the trained forests to use. For example, in Fig. 2, the forest trained on global variable values in bin 3 is selected. The global variable may be calculated by the test image, for example, by calculating the player height or in other ways.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Yoshihama in view of Furuya by selecting one of a plurality of trained models based on a height of the performer/player in the input image that is taught by Kohli, to make the invention that chooses a different model based on the height of the performer to output the performer-specific recommended information; thus, one of ordinary skilled in the art would be motivated to combine the references since taking into account one or more global variables (such as a person’s height) of the input image improves accuracy for many practical applications (Kohli, Paragraph [0023]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Naoyuki (Japanese Patent Application No. JP 2014-062945 A) teaches a music playing device that includes a musical sound generation control function to determine positions at which a player is hitting air musical instruments with playing members, from a relationship between position coordinates of the playing members in a captured video and position coordinates of positions corresponding to the air musical instruments, and generate musical sounds corresponding to the determined positions along with the movement of the playing members and a hitting point displacement detection function to detect displacement of the playing members to the air musical instruments.
Furuya et al. (U.S Patent No. 11,508,344 B2) teaches an information processing device, an information processing method and a program that provide, as feedback to a user, information based on a level of collaboration of multiple movement elements in a performance provided by collaborative or non-collaborative moves of multiple parts of a body of the user and that are usable to support mastering a performance.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7651. The examiner can normally be reached 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYDNEY L BLACKSTEN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
