DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-4, 6-7, 9, 13, 16-17, and 33 are amended. Claims 76-77 are canceled. Claims 78 and 79 are newly added. As such, claims 1-17, 33, and 78-79 are presented for examination.
Response to Arguments
Rejection under 35 U.S.C. 103
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 11-12, 14-16, and 78 are rejected under 35 U.S.C. 103 as being unpatentable over Thijssen et al. (US 20100293468 A1; hereinafter referred to as Thijssen) in view of Ashley et al. (US 20140168277 A1; hereinafter referred to as Ashley) and Cho et al. (US 20130010969 A1; hereinafter referred to as Cho).
Regarding claim 1, Thijssen discloses: a method performed by a computing device, comprising… the metadata including a size of a graphical object ([0032] audio settings associated with an application may be coupled to window settings (e.g., size and/or position of the window) associated with the application. For example, when a user increases (i.e., resizes) a size of the window, the volume may be increased. Conversely, when the user decreases the size of the window, the volume may be decreased) that comprises image data associated with the application ([0075] FIGS. 6A and 6B are diagrams illustrating an exemplary scenario related to controlling audio settings based on window settings. For example, as illustrated in FIG. 6A, assume that a window 605, associated with a media player, is providing multi-media content (e.g., a music video). Also see Fig. 1.);
presenting the graphical object… ([0037] FIG. 1 is a diagram illustrating an overview of an exemplary embodiment described herein. As illustrated, an exemplary window 105 may be displayed to a user on a display associated with a user device).
Thijssen does not explicitly, but Ashley teaches: maintaining metadata associated with an application running on the computing device… ([0061] position and size of the presented video, the audio level, the audio dynamic range, the ambient lighting level can all be modified in accordance with metadata associated with the presented content).
Thijssen and Ashley are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen to combine the teachings of Ashley because doing so would allow for audio parameters to be adapted in different ways according to different types of available metadata of an application, improving user flexibility in controlling audio output (Ashley [0059] the client device is operable to adapt the presentation of content according to several factors including content metadata; real-time analysis of the viewing environment 101; user control; etc. These factors will now be described in more detail).
The combination of Thijssen and Ashley does not explicitly, but Cho teaches: in response to determining that the size of the graphical object is not greater than a threshold ([0069] when the size of an image object is below a threshold value, the determination unit 240 may not provide a sound perspective to the sound object that corresponds to the image object. Since an image object having a very small size only slightly affects a user’s 3D effect experience, the determination unit 240 may optionally not provide any sound perspective to the corresponding sound object), rendering audio ([0077] When the determination unit 320 determines that sound perspective does not need to be provided to a sound section, the sound depth value in the corresponding sound section may be determined as a minimum value) associated with the application as a single audio channel based on a first dynamic range ([0121] When a user hears a reproduced mono signal from just one speaker, the user will typically not experience any stereoscopic sensation, but when the user hears a stereo signal reproduced by using at least two speakers, the user may experience a stereoscopic sensation) for playback through one or more speakers ([0104] A mixer 580 mixes at least one signal and outputs the mixed signal to a speaker or speaker system);
and in response to determining that the size of the graphical object ([0029] acquiring of the sound depth information includes carrying out the providing of the sound perspective at a level based on a size of each of the at least one image object) is greater than the threshold ([0082] the determination unit 320 may determine that sound perspective will be provided to a sound section that corresponds to an image frame only when Diff_Depth.sup.i is above a threshold value th), rendering the audio as a plurality of audio channels based on a second dynamic range for playback ([0129] sound perspective is provided to the sound object that corresponds to an image object, so that the sound object changes as it approaches the user. The user visibly senses that the image object jumps out of the screen and the user has the sensation that the sound object also approaches the user, thereby more realistically experiencing a stereoscopic sensation) through the one or more speakers ([0104] A mixer 580 mixes at least one signal and outputs the mixed signal to a speaker or speaker system).
Thijssen, Ashley, and Cho are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen and Ashley to combine the teachings of Cho because doing so would allow for the control of audio characteristics based on different thresholds of an image size, leading to more immersive user experience regarding images and corresponding audio of images, and greater user flexibility for presenting audio (Cho [0048] The perspective providing unit 130 processes a sound signal so that a user may sense or experience a sound perspective based on the sound depth information. The perspective providing unit 130 may provide the sound perspective according to each sound object after the sound objects corresponding to image objects are extracted, provide the sound perspective according to each channel included in a sound signal, or provide the sound perspective for all sound signals).
Regarding claim 2, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Thijssen further teaches: wherein the first dynamic range is less than the second dynamic range ([0037] the user may resize window 105 to adjust the volume. For example, when the user decreases the size of window 105 to a size corresponding to window 115, the audio associated with the video may be decreased. Conversely, when the user increases the size of window 115 to a size corresponding to window 120, the audio associated with the video may be increased).
Regarding claim 11, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Ashley further teaches: wherein the object is presented through an augmented reality, mixed reality, or virtual reality display ([0176] an augmented reality application running on companion device 137 could be used to overlay links to internet content when the companion device is pointed at the surface).
Thijssen, Ashley, and Cho are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen, Ashley, and Cho to further combine the teachings of Ashley because doing so would allow for more flexibility in different ways to interact with the graphical object to alter audio output, improving user experience (Ashley [0176] As well as being used to control the immersion level, and hence adapt the presentation of content, companion device 137 also enable interaction with content presented on the display surface).
Regarding claim 12, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Thijssen further teaches: wherein the graphical object is presented through a two-dimensional display ([0044] Display 225 may output visual content and may operate as an input component (e.g., a touch screen). For example, display 225 may include a liquid crystal display (LCD), a plasma display panel (PDP), a field emission display (FED), a thin film transistor (TFT) display, or some other type of display technology).
Regarding claim 14, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Thijssen further teaches: wherein the method is performed by an operating system (OS) of the computing device and the application is one of a plurality of applications managed by the OS ([0047] Processing system 305 may control the overall operation (or a portion thereof) of user device 200 based on an operating system and/or various applications), each of the plurality of applications being associated with corresponding metadata that includes a corresponding size of a corresponding graphical object ([0067-0068] In other implementations, the window setting/audio setting pair may be application-specific… when the window manipulation corresponds to increasing the size of the window, the audio setting may correspond to increasing the volume associated with the audio content).
Regarding claim 15, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 14. Thijssen further teaches: wherein, based on the corresponding size of the corresponding graphical object, an audio parameter that is associated with a corresponding one of the plurality of applications is determined ([0068] when the window manipulation corresponds to decreasing the size of the window, the audio setting may correspond to decreasing the volume associated with the audio content) and is applied to render audio associated with the corresponding one of the plurality of applications ([0070] The processed audio content may be output to speakers (block 530). Audio setter 420 may output the processed audio content associated with the window to speakers 215).
Regarding claim 16, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Thijssen further teaches: further comprising determining or modifying the first dynamic range or the second dynamic range based on the size of the graphical object that is contingent on a categorization of the application ([0057] Audio detector 410 may identify when application 315 provides audio content based on various factors, such as, for example, the type of application 315 or use or state information of application 315 (e.g., is application 315 currently playing audio content, is application 315 in a muted state, etc.)), the categorization including at least one of: media, communication, and utility ([0050] memory/storage 310 may include a variety of applications 315, such as, an e-mail application, a telephone application).
Regarding claim 78, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. Cho further teaches: wherein rendering the audio as the single audio channel comprises producing a single sound source through the one or more speakers ([0121] A user hears the sound object through at least one speaker. When a user hears a reproduced mono signal from just one speaker, the user will typically not experience any stereoscopic sensation, but when the user hears a stereo signal reproduced by using at least two speakers, the user may experience a stereoscopic sensation), wherein rendering the audio as the plurality of audio channels comprises producing a plurality of sound sources through the one or more speakers ([0125] the sound signal of a first channel is reproduced through a left speaker and the sound signal of a second channel is reproduced through a right speaker. A user may experience a stereoscopic sensation by hearing at least two sound signals generated from the different locations).
Claims 3-6 are rejected under 35 U.S.C. 103 as being unpatentable over Thijssen in view of Ashley and Cho, as applied to claims 1-2, 11-12, 14-16, and 78 above, and further in view of Zurek et al. (US 20150131966 A1; hereinafter referred to as Zurek).
Regarding claim 3, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. The combination of Thijssen, Ashley, and Cho does not explicitly, but Zurek discloses: further comprising determining the first dynamic range or the second dynamic range by generating corresponding audio filters ([0061] 3D video objects will have realistic and accurate sound parameters. In this context, task 406 may define the acoustic characteristics to account for parameters such as: acoustic impedance; acoustic reflection; sound absorption; acoustic dampening; frequency response; filtering; or the like) based on a model of a sound source that is associated with the object ([0034] the normalized 3D audio data may be subjected to at least one transformation that scales the 3D sound field in accordance with the particular size of the display element).
Thijssen, Ashley, Cho, and Zurek are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen, Ashley, and Cho to combine the teachings of Zurek because doing so would allow for audio to be filtered based on different sound sources, leading to better audio scaling for different sizes of an object (Zurek [0020] A much more efficient and practical approach (as described here) stores the audio data based on the individual 3D video objects to allow the host system to perform wave field synthesis calculations on the fly as needed. By performing the wave field synthesis calculation in the playback device, not only can the scaling issue of the source be accommodated, but also the wave field synthesis array can vary from playback device to playback device for the optimal sound reproduction for that size device while using the same content).
Regarding claim 4, the combination of Thijssen, Ashley, Cho, and Zurek teaches: the method of claim 3. Zurek further teaches: wherein a size or geometry of the model is determined based on the size or geometry of the graphical object ([0020] the 3D audio processing technique described here scales the virtual audio space in accordance with the virtual object space, and based on the dimensions of the display screen utilized by the presentation device, such that the acoustic source position aligns with the perceived visual object position).
Regarding claim 5, the combination of Thijssen, Ashley, Cho and Zurek teaches: the method of claim 4. Zurek further teaches: further comprising modifying the size or the geometry of the model in response to a change in the size or the geometry of the graphical object ([0046] the virtual audio space is scaled versus the virtual object space based on display size such that the acoustic source position aligns with the perceived visual object position).
Regarding claim 6, the combination of Thijssen, Ashley, Cho and Zurek teaches: the method of claim 3. Zurek further teaches: wherein one or more portions of the model radiates acoustic energy in simulation which determines a corresponding dynamic range ([0061] Task 406 defines acoustic characteristics such that the 3D video objects will have realistic and accurate sound parameters), the corresponding audio filters being generated from the acoustic energy ([0061] task 406 may define the acoustic characteristics to account for parameters such as: acoustic impedance; acoustic reflection; sound absorption; acoustic dampening; frequency response; filtering; or the like).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Thijssen in view of Ashley, Cho, and Zurek, as applied to claims 3-6 above, and further in view of Bharitkar (US 20200045493 A1).
Regarding claim 7, the combination of Thijssen, Ashley, Cho, and Zurek teaches: the method of claim 3. The combination of Thijssen, Ashley, Cho, and Zurek does not explicitly, but Bharitkar discloses: wherein the corresponding audio filters include a first filter associated with direct sound ([0016] The filters may be used, for example, for spatial rendering with direct sound and reflections using symmetric direct-sound HRTFs and reflections), a second filter associated with early reflections, and third filter associated with a reverberation, that are applied to the audio to render the audio ([0018] the filters may include crosstalk cancellers, spatial synthesizers, reflection filters, reverberation filters, etc.).
Thijssen, Ashley, Cho, Zurek, and Bharitkar are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen, Ashley, Cho, and Zurek to combine the teachings of Bharitkar because doing so would allow for audio to have higher quality spatial rendering and improve user experience (Bharitkar [0018] With respect to spatial rendering with speakers, various filters may be applied to an input audio signal to produce high-quality spatial rendering. For example, the filters may include crosstalk cancellers, spatial synthesizers, reflection filters, reverberation filters, etc.).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Thijssen in view of Ashley, Cho, and Zurek, as applied to claims 3-6 above, and further in view of Gibson (US 20030091204 A1).
Regarding claim 8, the combination of Thijssen, Ashley, Cho, and Zurek teaches: the method of claim 3. Zurek further teaches: generating of second audio filters based on the modified geometrical attributes of the model ([0061] if a 3D video object represents a character wearing soft clothing, then the acoustic characteristics may be defined such that the corresponding 3D audio appears to be muffled and has little to no associated sound reflections. In contrast, if a 3D video object represents a robot fabricated from sheets of metal, then the acoustic characteristics may be defined such that the corresponding 3D audio appears to be bright or tinny and has a high amount of associated sound reflections).
The combination of Thijssen, Ashley, Cho, and Zurek does not explicitly, but Gibson discloses: modifying geometrical attributes of the model based on user input ([0043] the visual images may then be manipulated and/or modified by the user in step 110, i.e. the visual characteristics of the visual images are altered, thereby causing corresponding changes to the audio signal in accord with the correlation scheme in step 106).
Thijssen, Ashley, Cho, Zurek, and Gibson are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen, Ashley, Cho, and Zurek to combine the teachings of Gibson because doing so would allow for audio to be controlled using image objects, providing greater flexibility and better visualization of the sound (Gibson [0009] The audio effect images have visual characteristics that correspond to audio effects added to the audio signals by the effects processing unit. A user control allows a user to adjust the displayed audio effect images. The effects processing unit changes audio effects added to the audio signals in response to corresponding user adjustments to the displayed audio effect images).
Claims 9-10 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Thijssen in view of Ashley and Cho, as applied to claims 1-2, 11-12, 14-16, and 78 above, and further in view of Gibson.
Regarding claim 9, the combination of Thijssen, Ashley, and Cho teaches: the method of claim 1. The combination of Thijssen, Ashley, and Cho does not explicitly, but Gibson teaches: further comprising determining, according to the size of the graphical object, one or more audio parameters that includes at least one of: a directivity pattern, a frequency response, a sound power, a frequency range, a pitch, a timbre, a number of output audio channels, and a reverberation ([0061] The duration of the Reverb (Reverb Time) is set in two ways. First, the size of the room is a function of the reverb time, as shown in FIGS. 10b and 10c. The reverb time may be lengthened by clicking anywhere on the cube and dragging up. Dragging down shortens the reverb time. Second, the reverb may be set by moving a slider 100 which also changes the size of the cube.).
Thijssen, Ashley, Cho, and Gibson are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Thijssen, Ashley, and Cho to combine the teachings of Gibson because doing so would allow for a user to change many different audio characteristics based on the size of an graphical object, improving user flexibility in manipulating audio using the object and visualizing changes (Gibson [0032] characteristics of the visual images, such as shape, size, spatial location, color, density and texture are correlated to selected audio characteristics, namely frequency, amplitude and time, such that manipulation of a visual characteristic causes a correlated response in the audio characteristic and manipulation of an audio characteristic causes a correlated response in the visual characteristic. Such a system is particularly well suited to showing and adjusting the masking of sounds in a mix).
Regarding claim 10, the combination of Thijssen, Ashley, Cho, and Gibson teaches: the method of claim 9. Gibson further teaches: further comprising, modifying at least one of the one or more audio parameters if the size of the graphical object is modified ([0043] Each visual image has a number of visual characteristics associated with it, such as size, location, texture, density and color, and these characteristics are correlated to audio signal characteristics of channel 12 in step 106).
Regarding claim 13, the combination of Thijssen, Ashley, Cho teaches: the method of claim 1. The combination of Thijssen, Ashley, Cho does not explicitly, but Gibson teaches: wherein rendering the audio comprises applying of the first dynamic range or the second dynamic range independently of user-controlled audio settings that are used to render the audio associated with the application ([0012] The audio signal visual image is segmented into portions that correspond to preselected frequency ranges. The frequency components of the selected audio signal are dynamically correlated with, and visually displayed with dynamic visual characteristics in, corresponding segmented portions of the audio signal visual image).
Claims 17 and 79 are rejected under 35 U.S.C. 103 as being unpatentable over Tatematsu et al. (US 20170353788 A1; hereinafter referred to as Tatematsu) in view of Ashley and Ishii et al. (US 20160328093 A1; hereinafter referred to as Ishii).
Regarding claim 17, Tatematsu teaches: a method performed by a computing device, comprising… the metadata including a size of a graphical object that comprises image data associated with the application ([0119] when enlarging or reducing the image displayed on the screen of display 163 in response to the zoom operation, computing device 151 changes sound directivity range ‘a to a size corresponding to a zoom magnification);
displaying, on a display screen coupled to the computing device ([0107] FIG. 13 and FIG. 14 each are a diagram illustrating the image displayed on the screen of display 163 and sound directivity range a, which vary depending on an input operation using mouse 168 by a user), a virtual environment of the application that includes the graphical object… ([0108] if the user performs the zoom-in operation around person 191 and 194 by using mouse 168, as shown in FIG. 13(B), the image displayed on the screen of display 163 is switched. The image of the vicinity of sofa 184 is enlarged and displayed on the switched screen. In addition, the directivity of a sound is formed in the direction corresponding to the center position of the image displayed on the switched screen, and sound directivity range a is set substantially at the center of sofa 184. In this case, the conversation between person 191 and person 194 sitting on sofa 184 is subjected to an emphasis process by computing device 151 and output from speaker 165. The graphical object can be an image of a person as shown in Fig. 13.);
responsive to the size not being greater than the threshold, determining a second directivity pattern for the graphical object that comprises a single monopole ([0103-0104] computing device 151 displays the image of the entire sound pickup area (target area) imaged by camera 10 on display 163, and outputs the sound picked up by microphone array 20 omnidirectionally from speaker 165 (S21)… Computing device 151 determines whether or not the zoom operation using mouse 168 is performed by the user (S22). In a case where the zoom operation is not performed, the process of computing device 151 returns to step S21. When zoomed out, the image is at a size threshold based on the zoom and therefore omnidirectional audio is output.);
and rendering audio associated with the graphical object ([0109] The image of person 191 sitting on the right side of sofa 184 is enlarged and displayed on the switched screen. In addition, a new directivity of a sound is formed in the direction corresponding to the center position of the image displayed on the switched screen, and sound directivity range a is set so as to cover person 191. In this case, the speaking voice of person 191 is subjected to an emphasis process by computing device 151 and output from speaker 165) according to the first directivity pattern or the second directivity pattern through one or more speakers ([0103] FIG. 12 is a flowchart illustrating an example of a forming operation procedure of directivity in a case where a zoom operation or a drag operation is performed in directivity control system 105 of Exemplary Embodiment 2. First, computing device 151 displays the image of the entire sound pickup area (target area) imaged by camera 10 on display 163, and outputs the sound picked up by microphone array 20 omnidirectionally from speaker 165 (S21)).
Tatematsu does not explicitly, but Ashley teaches: maintaining metadata associated with an application running on the computing device… ([0061] position and size of the presented video, the audio level, the audio dynamic range, the ambient lighting level can all be modified in accordance with metadata associated with the presented content).
Tatematsu and Ashley are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Tatematsu to combine the teachings of Ashley because doing so would allow for audio parameters to be adapted in different ways according to different types of available metadata of an application, improving user flexibility in controlling audio output (Ashley [0059] the client device is operable to adapt the presentation of content according to several factors including content metadata; real-time analysis of the viewing environment 101; user control; etc. These factors will now be described in more detail).
The combination of Tatematsu and Ashley does not explicitly, but Ishii teaches: responsive to the size being greater than a threshold, determining a first directivity pattern for the graphical object that comprises a plurality of lobes… ([0086] output control unit 122 may control the directivity of the sound according to the size of the display object Rs. Specifically, when the display object Rs is larger than a predetermined size, the output control unit 122 may have the sound output to the entire space inside the room from all of the loudspeakers. Sound from multiple loudspeakers represents multiple lobes.).
Tatematsu, Ashley, and Ishii are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Tatematsu and Ashley to combine the teachings of Ishii because doing so would allow for control of audio directivity and quality using image size thresholds, leading to greater control and flexibility for adjusting audio parameters (Ishii [0085] the output control unit 122 may control the sound quality according to the size of the display object Rs. Specifically, the output control unit 122 may control the equalizer settings according to the size of the display object Rs. More specifically, when the size of the display object Rs is larger than a threshold value, the output control unit 122 may set the equalizer settings to low sound emphasis, and when smaller than the threshold value, the equalizer settings may not be set to low sound emphasis).
Regarding claim 79, the combination of Tatematsu, Ashley, and Ishii teaches: the method of claim 17. Tatematsu further teaches: wherein a number of the plurality of lobes of the first directivity pattern is determined based on the graphical object and the size ([0100] Even when the image displayed on display 163 is switched by zoom-in or zoom-out, the size of sound directivity range a is fixed, such that if the image is enlarged by zoom-in, sound directivity range a also increases, and on the other hand, if the image is reduced by zoom-out, sound directivity range a also decreases (see FIG. 13(B) and FIG. 13(C)). Increasing the range of directivity increases the number of lobes.).
Claim 33 is rejected under 35 U.S.C. 103 as being unpatentable over Gibson in view of Cho and Bengtsson (US 20090216529 A1).
Regarding claim 33, Gibson discloses: a method performed by a computing device, comprising: maintaining metadata associated with an application running on the computing device ([0009] A microcomputer system includes a signal processing unit for transforming each audio signal into an audio signal visual image for display on a video display monitor. Each of the audio signal visual images has visual characteristics that correspond to the audio characteristics of the corresponding audio signal), the metadata including a size of a graphical object that comprises image data associated with the application ([0043] Each visual image has a number of visual characteristics associated with it, such as size, location, texture, density and color, and these characteristics are correlated to audio signal characteristics of channel 12 in step 106. Each channel which is either active or selected by the user is then displayed on the video display monitor 58 by showing the visual image corresponding to the channel in step 108. The visual images may then be manipulated and/or modified by the user in step 110, i.e. the visual characteristics of the visual images are altered);
presenting the graphical object… ([0039] The DSP unit 54 transforms each digitized sound signal into a visual image, which is then processed by CPU 52 and displayed on video display monitor 58. The displayed visual images may be adjusted by the user via user control 60);
and rendering the audio according to the first frequency response or the second frequency response for playback ([0032] The characteristics of the visual images, such as shape, size, spatial location, color, density and texture are correlated to selected audio characteristics, namely frequency, amplitude and time, such that manipulation of a visual characteristic causes a correlated response in the audio characteristic) through one or more speakers ([0010] A plurality of speakers broadcast the audio signals output[t]ed by the audio mixer).
Gibson does not explicitly, but Cho teaches: responsive to the size ([0029] The acquiring of the sound depth information includes carrying out the providing of the sound perspective at a level based on a size of each of the at least one image object) being greater than a threshold ([0069] when the size of an image object is below a threshold value, the determination unit 240 may not provide a sound perspective to the sound object that corresponds to the image object), determining a first frequency response of audio associated with the graphical object, wherein the first frequency response comprises a first low-frequency cut-off… ([0052] The perspective providing unit 130 adjusts the low-frequency band component of a sound object based on sound depth information. That is to say, a user may remarkably recognize the low-frequency band component in sounds perceived as being close by. Therefore, when the sound object is to be generated so as to be perceived as being close to the user, the low-frequency band component may be boosted).
Gibson and Cho are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Gibson to combine the teachings of Cho because doing so would allow for the control of audio characteristics based on different thresholds of an image size, leading to more immersive user experience regarding images and corresponding audio of images, and greater user flexibility for presenting audio (Cho [0048] The perspective providing unit 130 processes a sound signal so that a user may sense or experience a sound perspective based on the sound depth information. The perspective providing unit 130 may provide the sound perspective according to each sound object after the sound objects corresponding to image objects are extracted, provide the sound perspective according to each channel included in a sound signal, or provide the sound perspective for all sound signals).
The combination of Gibson and Cho does not explicitly, but Bengtsson teaches: responsive to the size not being greater than the threshold ([0062] the distance determination circuit 180 can include a circuit 184 that determines the size of a person's image in the image signal from the camera 116 and, responsive thereto, determines the distance between the terminal 110 and the person's image in the image stream from the camera 116 based on the size of the person's image. As shown, the image signal can also be routed to a video recorder 172, within the video and sound recorder 118, for recording therein. The circuit 184 is configured to recognize (operation 602) in the image signal the presence of a person, who may correspond to the targeted speaker or be proximately located to the targeted speaker. The circuit 184 can further estimate (operation 604) the distance from the terminal 110 to the person in response to the size of the person relative to a plurality of threshold values, where the threshold values may relate image size to expected distance), determining a second frequency response of the audio, which comprises a second low-frequency cut-off that is greater than the first low-frequency cut-off… ([0045] the adaptive sound filter circuit 150 includes a high-pass filter that filters the microphone signal 140. The high-pass filter has a cutoff frequency that is varied in response to the control signal 152. The speaker recognition circuit 162 can respond to recognition of the presence of the targeted speaker's voice in the microphone signal 140 by using the learned characteristics to determine a lower frequency range of the targeted speaker's voice and by regulating the control signal to vary the cutoff frequency of the high-pass filter in response to the determined lower frequency range).
Gibson, Cho, and Bengtsson are considered analogous in the field of audio processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Gibson and Cho to combine the teachings of Bengtsson because doing so would allow for control of audio frequency using image size thresholds, leading to greater control and flexibility for adjusting audio parameters (Bengtsson [0062] The circuit 184 can further estimate (operation 604) the distance from the terminal 110 to the person in response to the size of the person relative to a plurality of threshold values, where the threshold values may relate image size to expected distance. The circuit 184 may further account for a zoom setting of the camera 116 (when available) during the distance estimation. The speaker characterization circuit 164 and/or the speaker recognition circuit 162 can be further configured to adapt (operation 606) the learned characteristics of the targeted speaker's voice in response to the estimated distance).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nathan Tengbumroong whose telephone number is (703)756-1725. The examiner can normally be reached Monday - Friday, 11:30 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN TENGBUMROONG/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights