Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
This Final Office Action is responsive to Applicants’ amendments and arguments, as received 5/21/26. Claims 1-20 remain pending, of which claims 1, 14, and 18 are independent.
Claims 1-20 were previously rejected under 35 U.S.C. 112(b) as being indefinite. Based on Applicants’ reply, the aforementioned rejection is now withdrawn.
Allowable Subject Matter: as previously indicated, claims 13 and 17 are objected to as being dependent upon a base claim rejected with an art rejection, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office Action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
8. Claims 1-2, 4-6, 8-11, 14, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature “Acoustic Environment Transfer for Distributed Systems” (“Inventors”) in view of U.S. Patent No. 11853401 (“Nookula”).
Regarding claim 1, Inventors teach A method, comprising:
receiving, by a processor set, acoustic content data (page 3, Figure 1: “Content audio”, where the Figure 1 framework as discussed in the first full paragraph on page 4 is understood by the Examiner to be a computer-implemented framework that is conducive to machine-learning (e.g., as established by Inventors in their Introduction section on page 1, where the problem domain is discussed) and therefore would involve the use of basic computer elements to do the processing steps taught by the reference, e.g. using processor and memory implementations known in the state of the art and widely applied to the same problem domain);
receiving, by the processor set, acoustic data for a target environment (page 3, Figure 1: “Environment audio”);
training, by the processor set, a neural network model on the acoustic data for the target environment to extract features of the target environment (a style transfer model is trained (last paragraph of section 3’s introduction, near the bottom of page 4), and the feature transfer involved therein specifically encompasses the extraction of environment features and content features as discussed in the same section 3 introduction section on page 4 (e.g., the page’s 1st full paragraph), and that such extraction is performed via a convolutional neural network as discussed which would intuitively be subject to training before any meaningful or effective use);
transferring, by the processor set, the features of the target environment extracted by the neural network model to the acoustic content data (section 3’s introduction beginning at the bottom of page 3, discussing the transfer of style from an acoustic environment, as extracted (section 3 introduction section on page 4, i.e., the page’s 1st full paragraph)), to essentially one of many possible different target environments);
constructing, by the processor set, the acoustic content data with the transferred features of the target environment (section 3’s introduction section, in its 6th paragraph as found on page 4, discusses the generation of a new waveform with the Griffin-Lim algorithm, which the Examiner believes reads on the same sort of construction step recited here and as taught in [0047] of Applicants’ published specification).
The Inventors do not teach the further limitation for outputting, by the processor set, via a user interface (UI), information on and configurable options for the training of the neural network model on the acoustic data for the target environment. However, the notion of a GUI/UI framework to assist model design, development, tuning, and/or deployment is known, and it would be an obvious tool given in the existence in the greater state of the art relating to machine learning assistance to apply such a framework to facilitate the assembly of steps that Inventors’ reference teaches together in coordination. With that reasoning, the Examiner relies upon NOOKULA to teach what Inventors’ reference lacks, see, e.g., Nookula’s column 1 lines 7-35 (discussing the challenges in model development, generally), column 2 lines 10-61 (discussing the advantages to be realized by a GUI implementation that makes model building, training, development, etc. more accessible and convenient to users), and more specifically column 6 lines 8-32 and column 8 lines 33-38 (discussing more concretely how typical GUI elements can be used to facilitate typical model building aspects, such as those that define how the model is built, trained, tuned, and so forth – in other words, the exposure of a model’s configurable options to a user/designer/developer via a GUI such as Nookula contemplates).
Both references generally relate to the building of machine learning models, and training aspects thereto. Hence, they are generally related to a common field, and hence are analogous. It would have been obvious to one of ordinary skill in the art to package the steps and techniques taught by Inventors in their reference into a common framework such as Nookula’s, with a reasonable expectation of success, such as to realize the advantages Nookula discusses in model design and training and deployment and so forth that are made possible way of its GUI, and by extension apply them to the management of the modelling process taught by the Inventors.
Regarding claim 2, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation for outputting, via the UI, information on and configurable options for transferring features of the target environment to the acoustic content data (Nookula’s column 9 line 59 – column 10 line 7, and column 14 line 51 – column 15 line 11 for example, discussing a user’s capability to specify and define the “input data” for a model, e.g., by way of the reference’s GUI-driven approach, which the Examiner equates with a designer/developer practicing the Inventors’ taught framework having the ability to specify configuration option specification and feedback relating to its model design, use, training, calibration, etc.). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 4, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation for receiving, via the UI, user inputs to configure the configurable options for the training of the neural network model on the acoustic data for the target environment (Nookula’s column 9 line 59 – column 10 line 7, and column 14 line 51 – column 15 line 11 for example, discussing a user’s capability to specify and define the “input data” for a model, e.g., by way of the reference’s GUI-driven approach, which the Examiner equates with a designer/developer practicing the Inventors’ taught framework having the ability to specify configuration option specification and feedback relating to its model design, use, training, calibration, etc.). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 5, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation for generating one or more per-layer surrogates corresponding to one or more of the neural network layers (staying with section 3’s introduction in its 5th paragraph of the Inventors’ reference, where it is discussed that gram matrices are generated for captured features which intuitively correspond to layers of the CNN, and where the Examiner understands a gram matrix as discussed here to constitute a “surrogate” as recited based on Applicants’ own specification, see e.g., [0053]-[0058] of the published specification). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 6, Inventors in view of Nookula teach the method of claim 5, as discussed above. The aforementioned references further teach the additional limitation for generating the one or more per-layer surrogates comprises generating an average Gram matrix per layer for the one or more of the plurality of neural network layers (section 3’s introduction in its 5th paragraph of the Inventors’ reference, where it is discussed that gram matrices are generated for captured features which intuitively correspond to layers of the CNN, and where the Examiner understands a gram matrix as discussed here to constitute a “surrogate” as recited based on Applicants’ own specification, see e.g., [0053]-[0058] of the published specification). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 8, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation wherein training the neural network model on the acoustic data for the target environment comprises training a convolutional neural network (CNN) using training data and a filter specific to the target environment (trained style transfer network as discussed in the last paragraph of Inventors’ section 3 introduction, on page 4, where the style transfer network is understood to be a CNN as discussed therein, and further per section 5.1 of Inventors’ reference, see the declaration of the system as configured “with variable convolutional filter sizes in order to better capture the sound signatures from various environments”, which suggests to the Examiner that the filter fits the target environment). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 9, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation for enabling user inputs to select options from the information on and the configurable options for the training of the neural network model on the acoustic data for the target environment (Nookula’s column 9 line 59 – column 10 line 7, and column 14 line 51 – column 15 line 11 for example, discussing a user’s capability to specify and define the “input data” for a model, e.g., by way of the reference’s GUI-driven approach, which the Examiner equates with a designer/developer practicing the Inventors’ taught framework having the ability to specify configuration option specification and feedback relating to its model design, use, training, calibration, etc.). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 10, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references further teach the additional limitation wherein constructing the acoustic content data with the transferred features of the target environment comprises constructing the acoustic content data in accordance with:
xs = arg min x Ls (x, G˜(1), … , G˜(L)) = ∑ l=1 L {G˜(l) - G(l)(x) } F 2
G˜(l) = ∑i=1 N G(i)(xi) / N or G˜(l) = ∑i=1 N Gk(i) (xi) / N
where x and xs are embedding features of generated data and the target environment, respectively, Gk(i)(xi) is a low-rank approximation of G(i)(xi), and k controls the number of largest eigenvalues used for approximation (Inventors’ section 3’s introduction, in its 5th paragraph, found on page 4). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 11, Inventors in view of Nookula teach the method of claim 10, as discussed above. The aforementioned references further teach the additional limitation further comprising setting N=1 (Inventors’ section 3’s introduction, in its 5th paragraph, found on page 4) and performing a denoising task to separate semantic content and implicit style (Inventors’ section 5.4, in its 5th paragraph as found on page 7, discussing “to ensure the semantic content well-preserved, whilst the target style is transferred in the newly generated data.”). The motivation for combining the references is as discussed above in relation to claim 1.
Regarding claim 14, the claim includes the same or similar limitations as discussed above in relation to claim 1, and is therefore rejected under the same rationale. The Examiner notes that the present claim recites A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to practice the limitations of claim 1 as discussed above. The additional features bolded here are likewise taught by the references, see e.g., Inventors’ reference, where the Figure 1 framework as discussed in the first full paragraph on page 4 is understood by the Examiner to be a computer-implemented framework that is conducive to machine-learning (e.g., as established by Inventors in their Introduction section on page 1, where the problem domain is discussed) and therefore would involve the use of basic computer elements to do the processing steps taught by the reference, e.g. using processor and memory implementations known in the state of the art and widely applied to the same problem domain.
Regarding claim 18, the claim includes the same or similar limitations as discussed above in relation to claim 1, and is therefore rejected under the same rationale. The Examiner notes that the present claim recites A system comprising: a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to practice the limitations of claim 1 as discussed above. The additional features bolded here are likewise taught by the references, see e.g., Inventors’ reference, where the Figure 1 framework as discussed in the first full paragraph on page 4 is understood by the Examiner to be a computer-implemented framework that is conducive to machine-learning (e.g., as established by Inventors in their Introduction section on page 1, where the problem domain is discussed) and therefore would involve the use of basic computer elements to do the processing steps taught by the reference, e.g. using processor and memory implementations known in the state of the art and widely applied to the same problem domain.
9. Claims 3, 15, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Inventors in view of Nookula and further in view of U.S. Patent No. 10068557 (“Engel”).
Regarding claim 3, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references teach a UI, e.g. as discussed above in relation to claim 1, but not specifically a further limitation for outputting, via the UI, information on and configurable options for the constructing of the acoustic content data with the transferred features of the target environment. Rather, the Examiner relies upon ENGEL to teach what Inventors etc. otherwise lack, see e.g., Engel’s FIG. 14 as discussed per column 21 line 55 – column 22 line 8, teaching a UI that allows a user to mix embeddings to generate new audio waveforms, specifically by enabling the user to adjust aspects of the sounds (“interpolate”), in accordance with a framework that generally performs some of the same sound modelling and mixing aspects as Inventors as shown in Engel’s FIG. 12 for example.
The references generally relate to the building of machine learning models, and training aspects thereto. See, e.g., Inventors’ discussion of training aspects as discussed per claim 1, but also Engel’s FIG. 12. In particular, both of those references relate these machine learning aspects in the problem domain for sound/acoustic data. Hence, they are generally related to a common field, and therefore are analogous. It would have been obvious to one of ordinary skill in the art to extend the user’s manipulation of model aspects/parameters, as Inventors modified in view of Nookula permits, to include waveform presentation and control/adjustment, as Engel teaches, with a reasonable expectation of success, to provide users with a concrete way to control and manipulate the results of its modelling.
Regarding claim 15, the claim includes the same or similar limitations as claim 3 discussed above, and is therefore rejected under the same rationale.
Regarding claim 19, the claim includes the same or similar limitations as claim 3 discussed above, and is therefore rejected under the same rationale.
10. Claims 7, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Inventors in view of Nookula and further in view of Non-Patent Literature “Sound Texture Synthesis Using Convolutional Neural Networks” (“Caracalla”).
Regarding claim 7, Inventors in view of Nookula teach the method of claim 6, as discussed above. The aforementioned references teach a UI, e.g. as discussed above in relation to claim 1, but not specifically a further limitation for outputting the information on and the configurable options for the training of the neural network model on the acoustic data for the target environment comprises outputting the one or more per-layer surrogates. Rather, the Examiner relies upon CARACALLA to teach what Inventors etc. otherwise lack, see e.g., Caracalla’s 2.1.3 discussing the determination of per-layer Gram matrices, and their selection as parameters explicitly, as would be understood to define modelling and the training therefor.
The references generally relate to the building of machine learning models, and training aspects thereto. See, e.g., Inventors’ discussion of training aspects as discussed per claim 1, but also Caracalla’s section 2.1.1-2.1.2. In particular, both of those references relate these machine learning aspects in the problem domain for sound/acoustic data. Hence, they are generally related to a common field, and therefore are analogous. It would have been obvious to one of ordinary skill in the art to extend the user’s presentation and/or manipulation of model aspects/parameters, as Inventors modified in view of Nookula permits, to include additional parameters that are effective in defining the model’s design and training, as Caracalla teaches with gram matrices in its section 2.1.3, with a reasonable expectation of success, to provide users with more perspective and insight into the modelling.
Regarding claim 16, the claim includes the same or similar limitations as claim 7 discussed above, and is therefore rejected under the same rationale.
Regarding claim 20, the claim includes the same or similar limitations as claim 7 discussed above, and is therefore rejected under the same rationale.
11. Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Inventors in view of Nookula and further in view of U.S. Patent No. 10163429 (“Silverstein”).
Regarding claim 12, Inventors in view of Nookula teach the method of claim 1, as discussed above. The aforementioned references teach a UI, e.g. as discussed above in relation to claim 1, but not specifically a further limitation for outputting the information on and the configurable options for the training of the neural network model on the acoustic data for the target environment comprises enabling user-configurable options for a plurality of predefined generators. Rather, the Examiner relies upon SILVERSTEIN to teach what Inventors etc. otherwise lack, see e.g., Silverstein’s column 2 lines 22-44 and column 3 lines 42-59 discussing prior art instances that permit a user to visually modify controls in a UI that have the effect of calibrating a generator element in a sound mixing/design workflow.
The references generally relate to the building of machine learning models, and training aspects thereto. See, e.g., Inventors’ discussion of training aspects as discussed per claim 1, but also Engel’s FIG. 12. In particular, both of those references relate these machine learning aspects in the problem domain for sound/acoustic data. Hence, they are generally related to a common field, and therefore are analogous. It would have been obvious to one of ordinary skill in the art to extend the user’s manipulation of model aspects/parameters, as Inventors modified in view of Nookula permits, to include generator presentation and control/adjustment, as Silverstein teaches, with a reasonable expectation of success, to provide users with a concrete way to control and manipulate the results of its modelling.
Response to Arguments
Applicants’ arguments filed 5/21/26 have been fully considered but they are not persuasive:
On pages 10-14, Applicants generally argue that the applied secondary reference NOOKULA is not appropriate to combine with INVENTORS for a number of reasons, which the Examiner will address in turn below.
At this time, the reasons and the arguments are not persuasive to the Examiner, but the Examiner is willing to reconsider them if Applicants provide further clarification in response to these explanatory details provided by the Examiner below in response to Applicants’ arguments.
On page 12 of their Reply, Applicants begin with a characterization of Nookula that the Examiner fundamentally disagrees with:
PNG
media_image1.png
144
600
media_image1.png
Greyscale
At the outset, Applicants’ argument restricts Nookula’s breadth to essentially just backpropagation-training-oriented modelling. The Examiner disagrees. See, e.g., Nookula’s column 3 lines 15-32, which clearly extends its modeling UI approach to models such as ResNet, VGG, YOLO, and Tiny YOLO, in the space of image classification, and which the Examiner understands to be convolution-driven. Hence, these models, in the Examiner’s understanding, are not that dissimilar from the Inventors’ CNN model, even based on Applicants’ own characterization found at the very bottom of their Reply’s page 10.
Applicants are welcome to make a showing that these named models do not involve convolution, in which case the Examiner will reconsider this particular evidence in light of Applicants’ argument. The Examiner’s quick search of these models indicate that they are convolution-based. However, please see the next bullet point.
Further, and perhaps more concretely, Nookula’s paragraph beginning at column 6 line 42 discusses a similar extension of its modeling UI approach to a neural network that explicitly features “convolutional layers”, i.e. a CNN – again, as found in Inventors’ disclosure. Based on these teachings, Applicants’ argument that Inventors’ model is so dissimilar from what Nookula’s framework permits itself for application, that Nookula could not be used to facilitate the training of a model as taught by Inventors, is not persuasive.
Moreover, Applicants then also appear to argue that Nookula is inappropriate because it purportedly does not teach “the training of any neural network model on acoustic data, target-environment specific ranging, any configurable option whose operation is tied to acoustic data or a target environment.” The Examiner does not find this a compelling line of argument. The Examiner notes here: Nookula is not even being relied upon in the obviousness rejection for anything substantive to acoustic modelling specifically. Rather, Nookula is a flexible framework that allows users to pick from various model types or to custom build a model. In the Examiner’s understanding, Nookula can relate to different models, different data, etc. More on this later/below. Hence, it would appear to the Examiner that Applicants are arguing that the acoustic nature of the data per Inventors is so specialized that a broad and flexible framework such as Nookula’s framework cannot be applied to the type of modelling contemplated by Inventors. Such an argument would appear to unfairly limit Nookula’s applicability on the basis of underlying data subject matter. For example, steps such as a model selection, model parameter selection, and distillation of these things into items selectable from a menu are entirely agnostic to the type of data underlying the modelling. The Examiner discusses this difference in understanding as to the type of data per Nookula further below, in relation to Applicants’ argument about proper motivation to combine.
On page 13 of Applicants’ Reply, Applicants essentially argue that Nookula would have to be unreasonably modified to accommodate the modelling of Inventors’ model:
PNG
media_image2.png
596
610
media_image2.png
Greyscale
As the Examiner has already point out above, this argument is not persuasive, mostly because Applicants limit Nookula unreasonably. If Nookula is a flexible modelling UI that permits a user’s configuration of different types of models and also custom models, why would – as Applicants argue – certain UI controls mentioned in Nookula “have to removed”? Particularly in this era of UIs that allow different flavors of a particular work product to be made, the argument that selecting a particular type of work product necessitates removal of features not related to that same type has the effect of breaking the application/UI is not persuasive. This is especially true in view of Nookula’s stated flexibility to accommodate different model types, as the Examiner has already noted, and this is also especially true in view of Nookula’s stated capability within that flexibility to extend its advantages to a neural network having convolutional layers – like that found in Inventors’ disclosure, as the Examiner has already noted.
On page 14 of Applicants’ Reply, Applicants argue that there is no motivation to combine the Inventors and Nookula:
PNG
media_image3.png
216
610
media_image3.png
Greyscale
The Examiner disagrees with this. Applicants’ argument has the effect of limiting Nookula to only image classification modelling. However, in the paragraph starting at Nookula’s column 7 line 18, Nookula teaches that object detection, as relating to image classification, is a class of model for which many models exist for selection and use, and that the class of model (e.g., object detection) is first identified before the model instances in the library relating thereto are made available for selection and hence use. If a model class identification step as taught is present, then it suggests that there are many model classes beyond just object detection. Hence, for this reason, Applicants’ argument that Nookula only pertains to object detection / image classification is not persuasive.
Conclusion
THIS ACTION IS MADE FINAL. Applicants are reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHOURJO DASGUPTA whose telephone number is (571)272-7207. The examiner can normally be reached M-F 8am-5pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571 272 4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHOURJO DASGUPTA/Primary Examiner, Art Unit 2144
Prosecution Insights