DETAILED ACTION
[1] Remarks
I. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
II. Claims 1-14 are pending and have been examined, where claims 1-4, 6 and 9-14 is/are rejected, claim 5 and 7-8 is/are objected. Explanations will be provided below.
III. Inventor and/or assignee search were performed and determined no double patenting rejection(s) is/are necessary.
IV. Patent eligibility (updated in 2019) shown by the following: Claims 1-14 pass patent eligibility test because there is/are no limitation or a combination of limitations amounting to an abstract idea. Also, the following limitation or the combinations of the limitations: “extract, from a performance record, a portion corresponding to the target area, the performance record being obtained by capturing the plurality of players in a second playing of the piece of music or being obtained by receiving sounds produced in the second playing” effects a transformation or a reduction of a particular article to a different state or thing / adds a specific limitation(s) other than what is well-understood, routine and conventional in the field, or adding unconventional steps that confine the claim to a particular useful application and providing improvements to the technical field of deep learning, which recite additional elements that integrate the judicial exception into a practical application and amounting significant more.
[2] Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function. Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function.
Claim elements in this application that use the word “means” (or “step for”) are presumed to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Similarly, claim elements that do not use the word “means” (or “step for”) are presumed not to invoke 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
Claim(s) 1 is not interpreted under 35 U.S.C. 112(f) or pre-AIA U.S.C. 112 6th paragraph because of the following reason(s): limitations are modified by sufficient structure or material for performing the claimed function.
Claim(s) 2-14 does not require 35 U.S.C. 112(f) or pre-AIA U.S.C. 112 6th paragraph interpretation because they are method claims and / or they are CRM claims.
Upon examination of the specification and claims, the examiner has determined, under the best understanding of the scope of the claim(s), rejection(s) under 35 U.S.C. 112(a)/(b) is not necessitated because of the following reasons: sufficient support are provided in the written description / drawings of the invention.
[3] Grounds of Rejection
Claim Rejections - 35 USC § 103
1. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claims 1-3, 6, 9, 12 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over De Juan (US 20200097764) in view of Lin (J. -C. Lin et al., "Coherent Deep-Net Fusion To Classify Shots In Concert Videos," in IEEE Transactions on Multimedia, vol. 20, no. 11, pp. 3123-3136, Nov. 2018).
Regarding claim 1, De Juan discloses a performance recording system comprising: at least one memory configured to store instructions; and at least one processor (see figure 12, 1220, 1230 and 1240) configured to execute the instructions to:
determine a plurality of candidate regions in a target capture area of a camera by use of first captured imagery generated by the camera configured to capture a plurality of players in a first playing of a piece of music (see figure 3B illustration below, where each bounding box is read as the candidate regions);
select a target area from among the plurality of candidate regions (see figure 9, 985 illustration below):
PNG
media_image1.png
622
1351
media_image1.png
Greyscale
.
De Juan is silent in disclosing extract, from a performance record, a portion corresponding to the target area, the performance record being obtained by capturing the plurality of players in a second playing of the piece of music or being obtained by receiving sounds produced in the second playing.
Lin discloses extract, from a performance record, a portion corresponding to the target area, the performance record being obtained by capturing the plurality of players in a second playing of the piece of music or being obtained by receiving sounds produced in the second playing (see figure 1 illustration below, where the medium close up or medium long shot is read as performance record which also corresponds to the target area):
PNG
media_image2.png
276
738
media_image2.png
Greyscale
.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to include extract, from a performance record, a portion corresponding to the target area, the performance record being obtained by capturing the plurality of players in a second playing of the piece of music because to focus analysis on a specific player in order to perform a detailed review of performance and audio quality for specific individuals without needing to analyze the entire, complex scene.
Regarding claims 2 and 14 see the rationale and rejection for claim 1.
Regarding claim 3, De Juan discloses the performance recording method according to claim 2, wherein the plurality of players includes a player, and wherein the determining of the plurality of candidate regions includes:
detecting, from the first captured imagery, at least a portion of the player’s body and a musical instrument (see figure 3B illustration below); and
PNG
media_image3.png
410
774
media_image3.png
Greyscale
;
determining, based on the at least the portion of the player’s body and the musical instrument, at least one of the plurality of candidate regions (see paragraph 42, concept is annotated as “person” with specific instances of “person” also annotated, e.g., “conductor,” “singer,” and “violinist musician;” concept “musical ensemble” is annotated with specific musical instruments also annotated, e.g., “violin fiddle,” “music stand,” “viola,” and “music stand,” a larger bounding box encompasses the entire scene is annotated as “orchestra).
Regarding claim 6, De Juan discloses the performance recording method according to claim 2, wherein the determining of the plurality of candidate regions includes determining, based on a user-set image area in the first captured imagery, at least one of the plurality of candidate regions (see paragraph 42, concept is annotated as “person” with specific instances of “person” also annotated, e.g., “conductor,” “singer,” and “violinist musician;” concept “musical ensemble” is annotated with specific musical instruments also annotated, e.g., “violin fiddle,” “music stand,” “viola,” and “music stand,” a larger bounding box encompasses the entire scene is annotated as “orchestra,” the annotation is the user’s input, also see paragraph 46, the process proceeds to invoke the annotation solicitation unit 520 to obtains, at 585, annotations from a user).
Regarding claim 9, De Juan discloses the performance recording method according to claim 2, wherein the selecting of the target area includes selecting the target area based on user-selected imagery from among imagery represented in the plurality of candidate regions in second captured imagery representative of the plurality of players in the second playing (see paragraph 46, the process proceeds to invoke the annotation solicitation unit 520 to obtains, at 585, annotations from a user, see paragraph 42, concept is annotated as “person” with specific instances of “person” also annotated, e.g., “conductor,” “singer,” and “violinist musician, where the singer and violinist musician are read as the plurality of players in the second playing).
Regarding claim 12, De Juan discloses the performance recording method according to claim 2, wherein the performance record is second captured imagery generated by the camera in capturing the plurality of players in the second playing, and wherein the portion corresponding to the target area is imagery represented in the target area in the second captured imagery (see figure 3B illustration above, which includes plurality of players captured by a camera, figure 2A, 210-a includes a camera).
3. Claims 4, 10-11, and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over De Juan (US 20200097764) in view of Lin (J. -C. Lin et al., "Coherent Deep-Net Fusion To Classify Shots In Concert Videos," in IEEE Transactions on Multimedia, vol. 20, no. 11, pp. 3123-3136, Nov. 2018) and Wei (W. -L. Wei, J. -C. Lin, T. -L. Liu, H. -R. Tyan, H. -M. Wang and H. -Y. M. Liao, "Learning to Visualize Music Through Shot Sequence for Automatic Concert Video Mashup," in IEEE Transactions on Multimedia, vol. 23, pp. 1731-1743, publication date: 22 June 2020 on IEEE website).
Regarding claim 4, the combination with De Juan and Lin as a whole discloses all the limitations of claim 3, but is silent in disclosing the performance recording method according to claim 3, wherein the determining of the plurality of candidate regions includes: estimating, based on first audio obtained by receiving sounds produced in the first playing, an area in the target capture area in which the musical instrument is represented; and determining, based on the area in the target capture area in which the musical instrument is represented, at least one of the plurality of candidate regions.
Wei discloses the performance recording method according to claim 3, wherein the determining of the plurality of candidate regions includes: estimating, based on first audio obtained by receiving sounds produced in the first playing, an area in the target capture area in which the musical instrument is represented (see figure 1, illustration below where the target area is the image itself and the audio is employed to determine the type of shots); and determining, based on the area in the target capture area in which the musical instrument is represented, at least one of the plurality of candidate regions (see figure 1 illustration below, where the guitar is read as the musical instrument):
PNG
media_image4.png
341
1044
media_image4.png
Greyscale
.
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to include estimating, based on first audio obtained by receiving sounds produced in the first playing, an area in the target capture area in which the musical instrument is represented in order to estimating an area in the target capture area based on the first audio of a musical instrument is done to accurately locate the source, isolate it from other sounds, optimize microphone placement, creating a realistic stereo image.
Regarding claim 10, Wei discloses the performance recording method according to claim 2, wherein the extracting of the portion corresponding to the target area from the performance record includes extracting the portion corresponding to the target area from the performance record at a point of time dependent on a point of time of the selecting of the target area (see figure 1 illustration above, target area is the singer area, where the audio represents the performance record at a point of time). See the motivation for claim 4. In addition, extracting of the portion corresponding to the target area from the performance record which repeatedly isolating or focusing on that same target area.
Regarding claim 11, Wei discloses the performance recording method according to claim 2, wherein the extracting of the portion corresponding to the target area from the performance record includes extracting the portion corresponding to the target area from the performance record to a rhythm of the sounds produced in the second playing (see figure 1 illustration above, target area is the singer area, where the audio signal is read as the rhythm of the sounds produced in the second playing). See the motivation for claim 4. In addition, refining rhythmic precision, and allowing for the comparison of subtle variations.
Regarding claim 13, Wei discloses the performance recording method according to claim 2, wherein the performance record is second audio obtained by a microphone receiving the sounds produced in the second playing, and wherein the portion corresponding to the target area is part of the second audio, the part of the second audio being obtained by receiving sounds from the target area (see figure 1 illustration below, where the 1st audio is received by the audio captured with the video frame label extreme long shot and 2nd audio is received by the audio captured with the video frame labeled medium shot):
PNG
media_image5.png
226
835
media_image5.png
Greyscale
.
See the motivation for claim 4. In addition, the musician can simultaneously listen to and play along with a previously recorded track.
[4] Claim Objections
Claim(s) 5 and 7-8 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
With regards to claim 5, the examiner cannot find any applicable prior art providing teachings for the following limitation(s): detecting at least a portion of the player’s body and a musical instrument from the first captured imagery; setting, based on the at least the portion of the player’s body and the musical instrument, a plurality of image areas in the first captured imagery; and determining, based on a user-selected image area among the plurality of image areas, at least one of the plurality of candidate regions; in combination with the rest of the limitations of claims 1 and 2.
De Juan discloses the performance recording method according to claim 2, wherein the plurality of players includes a player, and wherein the determining of the plurality of candidate regions includes: detecting at least a portion of the player’s body and a musical instrument from the first captured imagery (see figure 3B illustration below on object detection using bounding box). De Juan also discloses the user requests for an abstract summary of the image, where the second type of query is to identify conceptually similar images, where the image including greed lawn, trees, and benches, the user requests to receive conceptually similar images (see paragraph 59), but does not disclosing setting, based on the at least the portion of the player’s body and the musical instrument, a plurality of image areas in the first captured imagery; and determining, based on a user-selected image area among the plurality of image areas, at least one of the plurality of candidate regions.
Regarding claim 7, the examiner cannot find any applicable prior art providing teachings for the following limitation(s): the performance recording method according to claim 2, wherein the selecting of the target area includes selecting the target area based on an amount of change in imagery represented in each of the plurality of candidate regions in second captured imagery representative of the plurality of players in the second playing; in combination with the rest of the limitations of claims 1 and 2.
Bradley (US 20180091743) discloses the performance recording method according to claim 2, wherein the selecting of the target area includes selecting the target area based on an amount of change in imagery represented in each of the plurality of candidate regions in second captured imagery (see paragraph 42, the motion features may include a variety of motion data, such as motion data obtained from one or more sensors, e.g., a gyroscope, motion data, such as optical flow magnitude, may also be used in determining whether to create an AutoLoop output video) but not representative of the plurality of players in the second playing.
Claim(s) 8 is/are objected as well because it is dependent on a claim with allowable subject matter.
CONTACT INFORMATION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEX LIEW (duty station is located in New York City) whose telephone number is (571)272-8623 (FAX 571-273-8623), cell (917)763-1192 or email alexa.liew@uspto.gov. Please note the examiner cannot reply through email unless an internet communication authorization is provided by the applicant. The examiner can be reached anytime.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MISTRY ONEAL R, can be reached on (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEX KOK S LIEW/Primary Examiner, Art Unit 2674 Telephone: 571-272-8623
Date: 3/4/26