DETAILED ACTION
A. This action is in response to the following communications: Transmittal of New Application filed 08/20/2024.
B. Claims 1-20 remains pending.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2022-029133, filed on 02/28/2022.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Eronen, Antti (US Pub. 2021/0287651 A1), herein referred to as “Eronen”.
As for claims 1, 11, 19 and 20, Eronen teaches. A sound production apparatus and corresponding method of claim 11 and 19 and product of claim 20 comprising:
an acquisition unit configured to acquire space information indicating a region of a three-dimensional virtual space including a sound source object and one or more three-dimensional objects having a first characteristic as an acoustic characteristic, and coordinate information indicating a configuration of the three-dimensional object (par. 62 VR virtual reality scenario representing VR space/ world and reproducing sound within space; par. 84 The audio scene 430 may comprise audio scene information 410 and audio information 420. The audio scene information 410 may comprise geometry, material properties, and/or other information related to the environment in which audio signals were captured or are to be experienced during rendering; par. 85 3DoF rendering of object, channel, and HOA content. In 3DoF rendering, the listener is able to listen to the audio scene at a single location while rotating their head in three dimensions);
a display control unit configured to display the three-dimensional virtual space (par. 62 Features as described herein generally relate to simulation of reverberation in rendering systems. An audio scene may be a captured or virtual audio scene, and may be rendered in an augmented reality (AR) or virtual reality (VR) scenario);
an input unit configured to input setting of a characteristic related to late reverberation of the three-dimensional virtual space (par. 87 AR or VR, may comprise determining the way in which early reflections and late reverberation change in different parts of the space as the listener moves in the virtual space. In addition, content creators or users may provide acoustic scenes, which may require methods for parameterizing the reverberation parameters of an arbitrary virtual space in a perceptually plausible way for those provided -acoustic scenes);
a change unit configured to change the first characteristic related to at least one of the three-dimensional objects to a second characteristic based on the characteristic related to late reverberation input by the input unit (par. 88 Reverberation refers to the persistence of sound in a space after the actual sound source has stopped. Different spaces are characterized by different reverberation characteristics. For conveying a spatial impression of an environment, reproducing reverberation in a perceptually plausible manner may be important; par. 94 included in the file parameters describing the desired reverberation characteristics; par. 99 terms “reverberator parameter” and “FDN parameter” may be used interchangeably); and
an output control unit configured to output in a switchable way a first reproduction sound synthesized based on the characteristic related to late reverberation and the first characteristic and a second reproduction sound synthesized based on the characteristic related to late reverberation and the second characteristic, the first reproduction sound and the second reproduction sound being a reproduction sound at a listening point of a sound emitted by the sound source object (par. 101 VR rendering, the reverberator parameters and/or reflection parameter(s) may be obtained based on an audio scene description format that may be used to perform early reflections and reverberation modeling (e.g. an encoder input format (EIF) definition), which may either contain reflection parameter(s) defined and indicated by a content creator or user, or determined by algorithms which process the scene geometry and derive relevant reflection parameter(s) automatically. Reverberator parameters may be optimized by algorithms based on reverberation characteristics included in the audio scene description format so that desired reverberation may be reproduced).
As for claim 2, Eronen teaches. The sound production apparatus according to claim 1, wherein the input unit inputs at least one of settings of a late reverberation level (par. 101 user defines parameters of reverberation), a delay time of late reverberation (par. 109 delay line lengths parameter), an attenuation time, a ratio of an attenuation amount for each frequency (par. 112 The attenuation filter coefficients, delay line length, and diffuse-to-direct ratio filter coefficients (i.e. reverberator parameters derived at 622) may be used, for example in a digital reverberator, to produce a diffuse reverberated signal, 656), an echo density, and a modal density as the characteristic related to late reverberation (par. 119 this selection of a higher number of delay lines may be based on the observation that complex scene geometries may require more delay lines to increase the pulse density; par. 122 modal density the rate at which the number of resonant modes occurs within a frequency band - The lengths of each delay line may be set according to standing wave resonance frequencies in the virtual room or physical room. The delay line lengths may further be made mutually prime. Additionally or alternatively, the delay line lengths may be based on the average mean free path of sound travel in the physical or virtual space, or may be otherwise related to the distance(s) between geometric elements in the space, between which reflection(s) and/or sound resonance(s) may occur.).
As for claim 3, Eronen teaches. The sound production apparatus according to claim 1, wherein the change unit changes a sound absorption coefficient set to the three-dimensional object as the acoustic characteristic related to the three-dimensional object (par. 97 Attenuation filter coefficients and delay line lengths for a feedback-delay-network (FDN) reverberator may be extracted/determined/optimized based on reverberation characteristics defined in the virtual scene description; these may be considered reverberator parameters).
As for claim 4, Eronen teaches. The sound production apparatus according to claim 3, wherein the change unit, in a case where an average sound absorption coefficient of the three-dimensional virtual space is calculated from a characteristic related to late reverberation input by the input unit, changes sound absorption coefficients of all the three-dimensional objects existing in the three-dimensional virtual space to the average sound absorption coefficient (par. 97 Attenuation filter coefficients and delay line lengths for a feedback-delay-network (FDN) reverberator may be extracted/determined/optimized based on reverberation characteristics defined in the virtual scene description; these may be considered reverberator parameters).
As for claim 5, Eronen teaches. The sound production apparatus according to claim 1, wherein the input unit, after the first characteristic related to at least one of the three-dimensional objects is changed to the second characteristic by the change unit, further inputs a change of the second characteristic to at least one of the three-dimensional objects existing in the three-dimensional virtual space via a user interface, and the change unit, in a case where the change of the second characteristic is input by the input unit, changes a characteristic of the three-dimensional object to which the change of the second characteristic is input by the input unit to a third characteristic (par. 110 audio scene may comprise one or more objects 614 and/or one or more channels 616. These objects or channels may have been captured in a real-world environment or may have been defined as part of a virtual environment to be rendered with the system 600. The one or more objects 614 and/or one or more channels 616 may be used to obtain one or more dry audio element signals, i.e. object and channel front end 658 at decoder/renderer 640) and changes a characteristic of another three-dimensional object to a fourth characteristic while maintaining setting of a characteristic related to late reverberation of the three-dimensional virtual space (par. 111 reverberator parameter(s) 622, extracted reflecting surface(s) 624, dry audio element signals 614 and/or 616, EIF scene description 612, and/or the output of the loudness analysis; par. 116 parameters related to the audio scene such as objects, channels, object positions, orientations and directivities, and their spatial extent; and decoding/rendering of the encoded audio scene 640, 690, which may be according to obtained reverberation parameters based on physical room information 730 and the encoded audio scene.).
As for claim 6, Eronen teaches. The sound production apparatus according to claim 5, wherein the input unit, in a case where the change of the second characteristic is input to at least one of the three-dimensional objects, changes a display mode of the three-dimensional object whose the second characteristic is changed to a display mode different from the other three-dimensional objects in the user interface (par. 117 - 121 discussion on different display modes (AR display and VR display) and how audio is reproduced in different display modes are handled differently).
As for claim 8, Eronen teaches. The sound production apparatus according to claim 5, wherein the output control unit, in a case where the change of the second characteristic is input by the input unit, outputs in a switchable way a reproduction sound synthesized before the change of the second characteristic is input by the input unit and a reproduction sound synthesized after the change of the second characteristic is input by the input unit, the reproduction sound being at the listening point of the sound emitted by the sound source object (par. 140-142 FIGS. 6 and 7, (or content creator/user) may determine the relevance or importance of different reflecting parameters; this can be represented dynamically on the display as a listener moves about geometry of audio space).
As for claim 9, Eronen teaches. The sound production apparatus according to claim 8, wherein the output control unit, in a case where the change of the second characteristic is input by the input unit, recalculates an early reflection sound at the listening point in the three-dimensional virtual space including the three-dimensional object having the third characteristic or the fourth characteristic, to synthesize a reproduction sound after the change of the second characteristic is input by the input unit (par. 140-142 FIGS. 6 and 7, (or content creator/user) may determine the relevance or importance of different reflecting parameters; this can be represented dynamically on the display as a listener moves about geometry of audio space).
As for claim 10, Eronen teaches. The sound production apparatus according to claim 1, wherein the output control unit outputs the first reproduction sound and the second reproduction sound by switching therebetween according to an operation of an operator in a user interface (par. 140-142 FIGS. 6 and 7, (or content creator/user) may determine the relevance or importance of different reflecting parameters; this can be represented dynamically on the display as a listener moves about geometry of audio space).
As for claim 12, Eronen teaches. The sound production method according to claim 11, wherein the output control step calculates, by parallel processing (par. 126-127 parallel graphic EQ filters), each of a direct sound from the sound source object to a listening point, a diffracted sound from the sound source object to a listening point, an early reflection sound calculated based on the second characteristic, and a late reverberation sound calculated based on the characteristic related to late reverberation, and synthesizes the reproduction sound based on a calculation result (par. 142; fig. 11 sound diffraction is indirectly discussed through the example of reflection from a sound source for purposes of illustrating the use of a beam tracing approach for simulating sound ray reflection in a virtual scene geometry. As depicted in fig. 11 there is sound diffraction by simulating obstacles in the environment; this environment as shown in figure 12 shows To synthesize an early reflection, the sound propagation from sound source may be traced via the reflecting parameters, for example reflection planes, to the listener; par. 149; par. 84 The audio scene information 410 may comprise geometry, material properties, and/or other information related to the environment in which audio signals were captured or are to be experienced during rendering; geometry in the scene causes diffracted sound at listener location).
As for claim 13, Eronen teaches. The sound production method according to claim 11, wherein the input step, after the first characteristic related to at least one of the three-dimensional objects is changed to the second characteristic by the change step, further inputs a change of the second characteristic to at least one of the three-dimensional objects existing in the three-dimensional virtual space via a user interface, and the change step, in a case where the change of the second characteristic is input, changes a characteristic of the three-dimensional object to which the change of the second characteristic is input to a third characteristic and changes a characteristic of another three-dimensional object to a fourth characteristic while maintaining setting of a characteristic related to late reverberation of the three-dimensional virtual space (par. 140-142 FIGS. 6 and 7, (or content creator/user) may determine the relevance or importance of different reflecting parameters; this can be represented dynamically on the display as a listener moves about geometry of audio space; This mechanism may allow essentially several descriptions (or “description levels”) to co-exist by using the importance to select what to consider in the rendering. This may give the scene author a mechanism to describe what is important (possibly for several levels of detail) so that each encoder and renderer may concentrate on making the most of the information that is most likely relevant to it (e.g. by making tradeoffs between reflection order and geometry “fidelity”); The same virtual scene comprising a corridor may be described with two levels of detail: (1) a higher level of detail with several “embeddings” in the walls (2) a lower level of detail that only has straight walls. One of these descriptions of the virtual scene may be indicated to a renderer, and the determination of relevant reflecting parameters may be performed accordingly, i.e. different reflecting parameters may be determined according to each description; sound sources may be moving or animated. Where the sound sources are moving sound sources, the movement trajectory of a moving sound source may be quantized to a fixed number of discrete positions, and the above procedure for simulating sound ray reflections and accumulating statistics of valid reflections may be repeated for the discrete positions; (par. 101 user defines parameters of reverberation; (par. 109 delay line lengths parameter) (par. 112 The attenuation filter coefficients, delay line length, and diffuse-to-direct ratio filter coefficients (i.e. reverberator parameters derived at 622)).
As for claim 14, Eronen teaches. The sound production method according to claim 13, wherein the output control step, in a case where the change of the second characteristic is input by the input step, outputs in a switchable way a reproduction sound synthesized before the change of the second characteristic is input by the input step and a reproduction sound synthesized after the change of the second characteristic is input by the input step, the reproduction sound being at the listening point of the sound emitted by the sound source object (par. 102 the encoding process may be iterative such that a first algorithmic step may produce an initial set of parameters which may then be utilized in a renderer to render reverberation for the content creator or user. The content creator or user may then listen to the output and adjust the parameters until a desired reverberation rendering quality is obtained; par. 140-142 the encoder, e.g. 640 of FIGS. 6 and 7, (or content creator/user) may determine the relevance or importance of different reflecting parameters. The importance may relate to the relevance of the reflecting parameters to different orders of early reflection rendering. For example, an importance (value/indication/indicator) for a reflecting parameter may indicate that this reflecting parameter is relevant).
As for claim 15, Eronen teaches. The sound production method according to claim 13, wherein the output control step generates a reproduction sound before the change of the second characteristic is input by the input step and a reproduction sound after the change of the second characteristic is input by the input step by parallel processing (par. 126-127 parallel graphic EQ filters), and outputs in a switchable way the generated reproduction sound (par. 140-142 creator/ user adjusts parameters and display updates dynamically based upon updates in parameters, wherein the user is able to move a listener around a geometric audio space and see audio beams change based upon properties of geometric objects in scene).
As for claim 16, Eronen teaches. The sound production method according to claim 15, wherein the output control step synthesizes a reproduction sound after the change of the second characteristic is input by the input step by using a direct sound and a diffracted sound calculated before the change of the second characteristic is input by the input step for synthesis of the reproduction sound after the change of the second characteristic is input by the input step (Par. 142-144 a sound source is moving along an animated trajectory, then the reflecting parameters, for example reflection planes, may change for different positions of the sound source trajectory. In an example embodiment, the reflecting parameters may be created and signaled, whether during encoding phase, decoding/rendering phase, or both, so that each sound source in the scene has its own set of reflecting parameters, or so that subsets of sound sources have separate set(s) of reflecting parameters).
As for claim 17, Eronen teaches. The sound production method according to claim 11, wherein the change step performs predetermined weighting based on information set in the three-dimensional object and changes the first characteristic to the second characteristic (par. 118 FIGS. 6 and 7, may be performed using a feedback delay network (FDN). Referring now to FIG. 8, illustrated is a FDN late reverberation generator 800 for rendering the diffuse late reverberation. The FDN 800 may take as input 810 a dry input signal and output a left output signal 852 and a right output signal 856 after processing based on one or more reverberator parameters, which may include attenuation filter coefficient(s), delay line length(s), and/or diffuse-to-direct ratio filter coefficient(s). In addition, the FDN 800 input parameters may include feedback matrix coefficients A, input gain coefficients b1 through bD, and/or output gain coefficients c1 through cD; par. 147 the reflecting parameters may be associated with different configurations of scene geometric elements. For example, an opening door in the scene may cause changes in potential reflection paths of sound rays in the scene and may change the relevance of different reflecting elements. Thus, the reflecting parameters may be associated to a state of geometric scene elements; for example, one set of reflecting parameters may be associated to the state of “door open,” whereas another set of reflecting parameters may be associated to the state of “door closed.”; par. 140-146 talks about making adjustments to the parameters based upon position of listener and the beam of audio that is interfered by geometric objects; based upon the object the parameters are handled differently thereby weighting is inferred because a sound parameter based upon reverberation is adjusted based upon at least position between listener and geometric object and what the object is, (e.g. door/wall etc…).
As for claim 18, Eronen teaches. The sound production method according to claim 17, wherein the change step performs predetermined weighting base on a name or shape of the three-dimensional object as information set in the three-dimensional object, and changes the first characteristic to the second characteristic (par. 147 the reflecting parameters may be associated with different configurations of scene geometric elements. For example, an opening door in the scene may cause changes in potential reflection paths of sound rays in the scene and may change the relevance of different reflecting elements. Thus, the reflecting parameters may be associated to a state of geometric scene elements; for example, one set of reflecting parameters may be associated to the state of “door open,” whereas another set of reflecting parameters may be associated to the state of “door closed.”; par. 140-146 talks about making adjustments to the parameters based upon position of listener and the beam of audio that is interfered by geometric objects; based upon the object the parameters are handled differently thereby weighting is inferred because a sound parameter based upon reverberation is adjusted based upon at least position between listener and geometric object and what the object is, (e.g. door/wall etc…).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Eronen in view of Norris, Glen A. (US Pub. 2017/0359467 A1), herein referred to as “Norris”.
As for claim 7, Eronen teaches. The sound production apparatus according to claim 6. Eronen does not teach three-dimensional object whose the second characteristic is changed in a color; however in the same field of endeavor Norris teaches wherein, the input unit displays the three-dimensional object whose the second characteristic is changed in a color different from the other three-dimensional object (par. 78-79 assist listeners in distinguishing between electronically generated binaural sound and physical environment sound by changing a color of an object this acts as a visual alert to the user, the color can also be dimming of a light, flashing etc…).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Norris into Eronen because Norris suggests challenges will arise with regard to how sound localization integrates into the modern era. Example embodiments offer solutions to some of these challenges and assist in providing technological advancements in methods and apparatus using 3D sound localization. Methods and apparatus assist listeners in distinguishing between electronically generated binaural sound and physical environment sound while the listener wears a wearable electronic device that provides the binaural sound to the listener. The wearable electronic device generates a visual alert or audio alert when the electronically generated binaural sound occurs (par. 2-3).
(Note :) It is noted that any citation to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006,1009, 158 USPQ 275, 277 (CCPA 1968)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
MIXED REALITY GAMING SYSTEM
Document ID
US 20200368616 A1
Date Published
2020-11-26
Abstract
An interactive mixed reality system for one or more users, in which both real-world entities and virtual world entities are capable of being interacted with by one or more users, or by objects (47, 91) for use by users, and the system arranged to computationally maintain game state and the evolution of events in the real-world and the virtual world, and the system arranged to generate a response (such as visual or tactile/haptic, or by way of cause and effect) which is experienced or perceived by the one or more users.
METHODS, SYSTEMS, AND COMPUTER READABLE MEDIA FOR ACOUSTIC CLASSIFICATION AND OPTIMIZATION FOR MULTI-MODAL RENDERING OF REAL-WORLD SCENES
Document ID
US 20180232471 A1
Date Published
2018-08-16
Abstract
Methods, systems, and computer readable media for acoustic classification and optimization for multi-modal rendering of real-world scenes are disclosed. According to one method for determining acoustic material properties associated with a real-world scene, the method comprises obtaining an acoustic response in a real-world scene. The method also includes generating a three-dimensional (3D) virtual model of the real-world scene. The method further includes determining acoustic material properties of surfaces in the 3D virtual model using a visual material classification algorithm to identify materials in the real-world scene that make up the surfaces and known acoustic material properties of the materials. The method also includes using the acoustic response in the real-world scene to adjust the acoustic material properties.
Inquires
Any inquiry concerning this communication should be directed to NICHOLAS AUGUSTINE at telephone number (571)270-1056.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
PNG
media_image1.png
213
559
media_image1.png
Greyscale
/NICHOLAS AUGUSTINE/Primary Examiner, Art Unit 2178 June 3, 2026
Prosecution Insights