Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Continued Examination Under 37 CFR 1.114
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/24/2025 has been entered. Claims 1, 8, and 15 have been amended. Claims 4, 11, and 18 have been canceled. Claims 1, 3, 5-8, 10, 12-15, 17, 19, 20, and 41-43 remain pending in the application.
Response to Amendments
3. Applicant’s amendments to claims 1, 8, and 15 have been fully considered and are persuasive. The amendments provided to overcome the 101 rejection (abstract idea) issued in the last office action is sufficient. The 35 U.S.C § 101 rejection of claims 1, 3, 5-8, 10, 12-15, 17, 19, 20, and 41-43 is respectfully withdrawn.
Response to Arguments
Applicant argues that Zhou describes the combination of audio objects with visual objects that are already existing in one or more scenes but are not associated with a spatial position and/or are associated with locations in multiple spatial environments. Therefore, Zhou does not teach or suggest at least one programmable circuit to insert a graphical object associated with the second audio object into the visual stream of the multimedia stream, as set forth in claim 1. Examiner respectfully disagrees and notes that Zhou is not limited to combining sound objects with already existing visual objects. Zhou expressly teaches determining whether a sound object corresponds to a video object and in response to determining that a sound object with known positional information does not corresponding to any video object, using the known positional information of the sound object to help identifying/generating/enhancing a corresponding video object (para. [0078-0080]). Thus, Zhou teaches the claimed feature of determining that an audio object is not correlated with visual objects and, in response, generating/enhancing a corresponding video object. Furthermore, it is also noted that the claim does not exclude visual objects that already exist in one or more scenes. The claim only requires that the second audio object is not correlated with one or more visual objects in the visual stream.
Claim Rejections – 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 1, 3, 5, 8, 10, 12, 15, 17, 19, 41, and 42 are rejected under 35 U.S.C. 103 as being unpatentable over Stein (U.S. Patent Application Pub. No. US 20220345813 A1) in view of Younessian (U.S. Patent Application Pub. No. US 20220130408 A1), and further in view of Zhou et al. (U.S. Patent Application Pub. No. US 20170364752 A1).
Claim 1: Stein teaches an apparatus comprising:
at least one memory (i.e. memory; para. [0108]);
instructions (i.e. FIG. 9 is a diagrammatic representation of a machine 900 within which instructions 908 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 900 to perform any one or more of the methodologies discussed herein can be executed; para. [0105]); and
at least one programmable circuit to execute the instructions to at least (i.e. a field programmable gate array (FPGA) or other programmable logic device … the various memories (e.g., memory 904, main memory 912, static memory 914, and/or memory of the processors 902) and/or storage unit 916 can store one or more instructions or data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 908), when executed by processors or processor circuitry; para. [0102, 0114]):
detect a first visual object in a visual stream of a multimedia stream, the first visual object associated with a first location (i.e. the second object 212 can have a location that can be designated by coordinates, such as radial or spherical coordinates, and can include information about a distance and an angle (e.g., including azimuth and/or elevation) from the reference position 201 or from a reference direction, such as the Look Direction. In the example of FIG. 3, the second object 212 can have a location that is defined by the radius R1, an azimuth of 0°, and an elevation of 0°; para. [0057]) in a content creation space represented by the multimedia stream (i.e. the processor circuit 410 can include an object classifier module 402. The object classifier module 402 can be configured to implement one or more aspects of the classifier circuit discussed herein. For example, the object classifier module 402 can be configured to receive image or depth information from the depth sensor 230 and apply artificial intelligence-based tools, such as machine learning or neural network-based processing, to identify one or more physical objects that are present in an environment; para. [0023, 0067]);
convert an audio stream of the multimedia stream into a frequency domain (i.e. the processor circuit 410 includes an FFT module 404 configured to receive audio signal information from the audio capture device 220 and convert the received signal (s) to the frequency domain. The converted signal can be processed using spatial processing, steering, or panning to change a location, depth, or frame of reference for the received audio signal information; para. [0066]);
detect a first audio object in the audio stream of the multimedia stream via analysis of the audio stream in the frequency domain (i.e. the processor circuit 410 includes a spatial analysis module 406 that is configured to receive the frequency domain audio signals from the FFT module 404 and, optionally, receive at least a portion of the audio data associated with the audio signals. The spatial analysis module 406 can be configured to use a frequency domain signal to determine a relative location of one or more signals or signal components thereof; para. [0068, 0080]), the first audio object associated with a second location in the content creation space (i.e. the processor circuit 410 can include a signal forming module 408. The signal forming module 408 can be configured to use a received frequency domain signal to generate one or more virtual sources that can be output as sound objects with associated metadata, or can be encoded as a spatial audio signal. In an example, the signal forming module 408 can use information from the spatial analysis module 406 to identify or place the various sound objects in respective designated locations or at respective depths in a soundfield; para. [0070, 0071, 0080]);
evaluate a correlation between the first visual object and the first audio object, the correlation based on the first location and the second location (i.e. the signal forming module 408 can use results or products of the spatial analysis module 406 with information from the depth sensor 230, optionally processed with the object classifier module 402, to determine audio source locations or depths. For example, the signal forming module 408 can use correlation information or can determine whether a correlation exists between identified physical objects or depths in the image data and audio information received from the spatial analysis module 406. In an example, determining a correlation can be performed at least in part by comparing the directions or positions of the identified visual objects with those of the identified audio objects. Other modules or portions of the processor circuit 410 can similarly or independently be used to determine correlations between information in the image data and information in the audio data; para. [0072, 0084]);
detect a second visual object in the visual stream (i.e. FIG. 2A, the first environment 210 includes various objects at respective various depths relative to the reference position 201. The first environment 210 includes some objects that may generate or produce sound and others that may not. For example, the first environment 210 includes a first object 211, such as a squeaking duck toy, and a second object 212, such as a roaring lion toy. The first environment 210 can include other objects such as a color panel, boxes, canisters, and more; para. [0053, 0054]);
detect a second audio object in the audio stream (i.e. The spatial analysis module 406 can be configured to use a frequency domain signal to determine a relative location of one or more signals or signal components thereof. For example, the spatial analysis module 406 can be configured to determine that a first sound source is or should be positioned in front (e.g., 0° azimuth) of a listener or a reference video location and a second sound source is or should be positioned to the right (e.g., 90° azimuth) of the listener or reference video location; para. [0068, 0070]);
detect a second audio object in the audio stream; based on a determination that the second audio object is not correlated with one or more visual objects in the visual stream (i.e. the signal forming module 408 can use results or products of the spatial analysis module 406 with information from the depth sensor 230, optionally processed with the object classifier module 402, to determine audio source locations or depths. For example, the signal forming module 408 can use correlation information or can determine whether a correlation exists between identified physical objects or depths in the image data and audio information received from the spatial analysis module 406. In an example, determining a correlation can be performed at least in part by comparing the directions or positions of the identified visual objects with those of the identified audio objects. Other modules or portions of the processor circuit 410 can similarly or independently be used to determine correlations between information in the image data and information in the audio data; para. [0072, 0096]);
generate metadata for the multimedia stream based on the correlation between the first visual object and the first audio object (i.e. In examples with high correspondence or correlation, the signal forming module 408 can use a weighted combination of positional information from the audio and visual objects. For example, weights can be used to indicate a relative direction of audio objects that will best match a spatial audio distribution, and can be used together with depth information from the depth sensor visual data or image data. This can provide a final source position encoding that most accurately matches depth capabilities of the spatial audio signal output to the acoustic environment in which the depth sensor and audio capture device are used; para. [0070-0073]); and
modify the multimedia stream based on the metadata (i.e. In an example, signals from the signal forming module 408 can be provided to other downstream processing modules that can help generate signals for transmission, reproduction, or other processing. For example, the spatial audio signal output from the signal forming module 408 can include or use virtualization processing, filtering, or other signal processing to shape or modify audio signals or signal components. The downstream processing modules can receive data and/or audio signal inputs from one or more modules and use signal processing to rotate or pan the received audio signals; para. [0074]).
Stein does not explicitly teach based on a determination that the second visual object is not correlated with one or more audio objects in the audio stream including the first audio object, insert a sound effect into the audio stream of the multimedia stream; insert a graphical object associated with the second audio object into the visual stream of the multimedia stream.
However, Younessian teaches detect a second visual object in the visual stream (i.e. an output module 123 (e.g., an output layer, a softmax layer, a rectified linear unit (ReLU), etc.) may generate/produce an output associated with one or more portions (e.g., one or more frames, etc.) of the visual content 103, such as a distribution of visual elements (e.g., objects, actions, scenes, events, etc.) associated with the plurality of portions of the visual content 103; para. [0036]); based on a determination that the second visual object is not correlated with one or more audio objects in the audio stream including the first audio object (i.e. At 530, a candidate auditory event of the one or more candidate auditory events may be determined to be missing from the audio content associated with the portion of the content item. For example, each auditory event included with the audio content may be compared, based on the timecode, to a candidate auditory event of the one or more candidate auditory events to determine an auditory event that is missing from the one or more auditory events; para. [0088]), insert a sound effect (i.e. A content item (e.g., multimedia content, a movie, streaming content, etc.) may include audio content (e.g., sounds, sound effects, speech/dialogue, etc.) that corresponds to different portions (e.g., scenes, frames, etc.) of the content item. The audio content may include one or more auditory events that provide context to specific scenes and/or events depicted by the content item and/or aid subjective perception (e.g., what is seen, heard, interpreted, etc.) of the content item. An auditory event may include a sound, a plurality of sounds, a sound effect, a voice, music, and/or the like; para. [0023]) into the audio stream of the multimedia stream (i.e. The audio content 102 may be modified to include one or more additional auditory events, such as auditory events indicated by the distribution of candidate auditory events 126 associated with the content item 101 that are not included with the distribution of auditory events 134; para. [0055, 0058]); generate metadata for the multimedia stream based on the correlation between the first visual object and the first audio object (i.e. The system 100 may include a correlation unit 140. The correlation unit 140 may determine correlations between a distribution of media elements, such as the distribution of visual elements 124 and/or the distribution of textual elements 115, and one or more auditory events, such as auditory events stored in an auditory event repository 150. For example, the correlation unit 140 may receive the distribution of visual elements 124 and compare elements of the distribution of visual elements 124 to stored/predefined auditory events described and/or include in an auditory event repository 150; para. [0046, 0053, 0055]); and modify the multimedia stream based on the metadata (i.e. At 540, the audio content may be modified. The audio content may be modified to include the candidate auditory event. For example, a sound (e.g., a raw waveform, a sound clip, an audio file, etc.) of and/or associated with the candidate auditory event determined to be missing from the one or more auditory events may be obtained, for example, from the auditory event repository. An audio file associated with the audio content may be modified/augmented to include the sound (e.g., a raw waveform, a sound clip, an audio file, etc.) of the candidate auditory event determined to be missing from the one or more auditory events; para. [0058-0061, 0089]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Stein to include the feature of Younessian. One would have been motivated to make this modification because it ensures seamless multimedia experiences and enhances audio-visual coherence.
However, Zhou teaches detect a second audio object in the audio stream (i.e. generates a plurality of sound objects (e.g., object size information, object location information, sound volume, audio sample data, etc.) from the audio data as captured by the microphone system; para. [0059, 0063, 0075]); based on a determination that the second audio object is not correlated with one or more visual objects in the visual stream including the first visual object (i.e. In response to determining that a sound object with known object positional information does not correspond to any video object with known object positional information; para. [0078, 0079, 0080]), insert a graphical object associated with the second audio object into the visual stream of the multimedia stream. (i.e. a media system may combine first sound objects and first video objects derived from a first spatial environment (or a first scene) with second sound objects and second video objects derived from one or more second spatial environments (or second scenes) into an overall scene. Additionally, optionally, or alternatively, a media system may combine sound objects (e.g., human voice, ambient sound, etc.) and/or visual objects (e.g., computer-generated graphics, human faces, etc.) that are not necessarily associated with any spatial positions with sound objects (e.g., a human voice, etc.) and/or visual objects (e.g., computer-generated graphics, human faces, etc.) that are located in specific spatial positions in one or more spatial environments. The media system may assign a sound object or a visual object that does not have a spatial position to a specific spatial position (e.g., a person speaking from behind a door, etc.), for example, based on user input. The media system may also assign a sound object or a visual object that has a first spatial position to a second different spatial position, for example, based on user input; para. [0095, 0114])
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein and Younessian to include the feature of Zhou. One would have been motivated to make this modification because it ensures a more cohesive and synchronized multimedia experience.
Claim 3: Stein, Younessian, and Zhou teach the apparatus of claim 1. Stein further teaches wherein one or more of the at least one programmable circuit (i.e. The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor, a processing device, a computing device having one or more processing devices, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device; para. [0102]) is to determine the sound effect based on a classification of the second visual object (i.e. At step 840, the fourth method 800 can include classifying the objects identified at step 830. In an example, step 840 can include using a neural network-based classifier or machine learning classifier to receive image information and, in response, provide a classification for the identified objects. The classifier can be trained on various data, for example, to recognize humans, animals, inanimate objects, or other objects that may or may not produce sounds. Step 850 can include determining audio characteristics associated with a classified object. For example, if a human male is identified at step 840, then step 850 can include determining an acoustic profile that corresponds to a human male voice, such as can have various frequency and transience characteristics; para. [0099, 0100]).
Younessian further teaches wherein one or more of the at least one programmable circuit is to determine the sound effect based on a classification of the second visual object (i.e. FIG. 2 shows a table 200 of example auditory events/labels that may be used to determine candidate auditory events. The table 200 lists auditory events/labels of various classifications/categories that may be stored by the auditory event repository 150. For example, column 201 lists auditory events/labels classified/categorized as animal sounds, column 202 lists auditory events/labels classified/categorized as natural soundscapes & water sounds, column 203 lists auditory events/labels classified/categorized as human sounds (non-speech), column 204 lists auditory events/labels classified/categorized as interior/domestic sounds, and column 205 lists auditory events/labels classified/categorized as exterior/urban sounds. The auditory event repository 150 may include auditory events that may be associated with any classification and/or category; para. [0049]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein and Zhou to include the feature of Younessian. One would have been motivated to make this modification because it ensures seamless multimedia experiences and enhances audio-visual coherence.
Claim 5: Stein, Younessian, and Zhou teach the apparatus of claim 1. Stein further teaches wherein the audio stream is a first audio stream, and one or more of the at least one programmable circuit (i.e. The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor, a processing device, a computing device having one or more processing devices, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device; para. [0102]) is to, based on a spatial relationship between the first location and the second location (i.e. the signal forming module 408 can use results or products of the spatial analysis module 406 with information from the depth sensor 230, optionally processed with the object classifier module 402, to determine audio source locations or depths. For example, the signal forming module 408 can use correlation information or can determine whether a correlation exists between identified physical objects or depths in the image data and audio information received from the spatial analysis module 406. In an example, determining a correlation can be performed at least in part by comparing the directions or positions of the identified visual objects with those of the identified audio objects. Other modules or portions of the processor circuit 410 can similarly or independently be used to determine correlations between information in the image data and information in the audio data; para. [0072]): identify a microphone associated with the first visual object (i.e. a system to perform spatial audio capture with depth can include a microphone array or soundfield microphone configured to capture a soundfield or sound scene. The system can include a depth camera or depth sensor configured to determine or estimate a depth of one or more objects in a field of view of the sensor, and can optionally be configured to receive depth information from multiple directions (e.g., up/down, left/right, etc.). In an example, the system can augment captured acoustic information with depth or distance information received from a depth sensor, and then encode the acoustic information and depth information in a spatial audio signal. The spatial audio signal can include components or sources with respective depths or distances relative to an origin or reference location; para. [0025, 0048]); and identify an association between the first visual object and a second audio stream of the multimedia stream, the second audio stream associated with the microphone (i.e. the signal forming module 408 can be configured to use information from both the spatial analysis module 406 and the object classifier module 402 to identify or place the various sound objects identified by the spatial analysis module 406. In an example, the signal forming module 408 can use information about identified physical objects or audio objects, such as information about an acoustic profile or signature of an identified object, to determine whether the audio data (e.g., received using the audio capture device 220) includes information that corresponds to the acoustic profile; para. [0071, 0072]).
Claim 8 is similar in scope to Claim 1 and is rejected under a similar rationale.
Stein teaches at least one non-transitory computer readable medium (i.e. Any one or more of the disclosed circuits or processing tasks can be implemented or performed using a general-purpose machine or using a special, purpose-built machine that performs the various processing tasks, such as using instructions retrieved from a tangible, non-transitory, processor-readable medium; para. [0104]) comprising computer readable instructions (i.e. FIG. 9 is a diagrammatic representation of a machine 900 within which instructions 908 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 900 to perform any one or more of the methodologies discussed herein can be executed; para. [0105]) to cause at least one programmable circuit to at least (i.e. The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor, a processing device, a computing device having one or more processing devices, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device; para. [0102]).
Claims 10 and 12 are similar in scope to Claims 3 and 5 and are rejected under a similar rationale.
Claims 15, 17, and 19 are similar in scope to Claims 1, 3, 5 and are rejected under a similar rationale.
Claim 41: Stein, Younessian, and Zhou teach the apparatus of claim 5. Stein does not explicitly teach wherein the sound effect is a Foley sound effect.
Younessian further teaches wherein the sound effect is a Foley sound effect (i.e. The features may be labeled with appropriate filter information (e.g., music, speech/dialogue, footsteps, siren, breathing, clapping, crying, etc.) to create one or more labeled data sets. The one or more labeled data sets may be used to train a deep neural network module 132 to detect auditory events (e.g., specific sounds, etc.), such as speech/dialogue, footsteps, siren, breathing, clapping, crying, and/or the like, in each portion of the audio content 102; para. [0042, 0048, 0049]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein and Zhou to include the feature of Younessian. One would have been motivated to make this modification because it ensures seamless multimedia experiences and enhances audio-visual coherence.
Claim 42: Stein, Younessian, and Zhou teach the apparatus of claim 5. Stein does not explicitly teach to modify the multimedia stream based on the metadata by inserting a label into a video stream of the multimedia stream.
Younessian further teaches to modify the multimedia stream based on the metadata by inserting a label into a video stream of the multimedia stream (i.e. the combined distribution of media elements 125 may include both a “police car” label/description and a “siren” label/description in a first portion of the combined distribution of media elements 125 that corresponds to a first timecoded portion of the content item 101 based on the corresponding first portions of the distribution of visual elements 124 and the distribution of textual elements 115, respectively. The combined distribution of media elements 125 may include a “fire” label/description in the second portion of the combined distribution of media elements 125 that corresponds to a second timecoded portion of the content item 101 based on the duplicated label/description of “fire” corresponding to the second portions of the distribution of visual elements 124 and the distribution of textual elements 115; para. [0046, 0047, 0053]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein and Zhou to include the feature of Younessian. One would have been motivated to make this modification because it ensures seamless multimedia experiences and enhances audio-visual coherence.
6. Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Stein in view of Younessian, Zhou, and further in view of Wexler et al. (U.S. Patent Application Pub. No. US 20230005471 A1).
Claim 6: Stein, Younessian, and Zhou teach the apparatus of claim 5. Stein further teaches wherein one or more of the at least one programmable circuit (i.e. The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a general purpose processor, a processing device, a computing device having one or more processing devices, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device; para. [0102]) is to enhance the second audio stream by audio associated with the first audio object (i.e. signals from the signal forming module 408 can be provided to other downstream processing modules that can help generate signals for transmission, reproduction, or other processing. For example, the spatial audio signal output from the signal forming module 408 can include or use virtualization processing, filtering, or other signal processing to shape or modify audio signals or signal components. The downstream processing modules can receive data and/or audio signal inputs from one or more modules and use signal processing to rotate or pan the received audio signals; para. [0074]).
Stein does not explicitly teach amplifying audio.
However, Wexler teaches wherein one or more of the at least one programmable circuit is to enhance the second audio stream by amplifying audio associated with the first audio object (i.e. processor 210 may cause selective conditioning of audio associated with individual 2310. The conditioning may include amplifying audio signals determined to correspond to sound 2421 (which may correspond to a voice of individual 2310) relative to other audio signals. In some embodiments, amplification may be accomplished digitally, for example by processing audio signals associated with sound 2421 relative to other signals. Additionally, or alternatively, amplification may be accomplished by changing one or more parameters of microphone 1720 to focus on audio sounds associated with individual 2310. For example, microphone 1720 may be a directional microphone and processor 210 may perform an operation to focus microphone 1720 on sound 2421. Various other techniques for amplifying sound 2421 may be used, such as using a beamforming microphone array, acoustic telescope techniques, etc.; para. [0198]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein, Younessian, and Zhou to include the feature of Wexler. One would have been motivated to make this modification because it enhances the overall multimedia experience by highlighting important audio elements.
Claims 13 and 20 are similar in scope to Claim 6 and are rejected under a similar rationale.
7. Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Stein in view of Younessian, Zhou, and further in view of Adsumilli et al. (U.S. Patent Application Pub. No. US 20170366896 A1).
Claim 7: Stein, Younessian, and Zhou teach the apparatus of claim 1. Stein further teaches wherein the first location is determined (i.e. the depth sensor 230 can include a system with a transmitter and a receiver, and can be configured to use active sampling techniques to deter mine an object depth. For example, the transmitter can emit a signal and use timing information about a bounce-back signal to establish, e.g., a point-cloud representation of an environment. The depth sensor 230 can include or use two or more sensors, such as passive sensors, that can concurrently receive information from an environment and from different perspectives. Parallax in the received data or images can be used to determine depth information about various objects in the environment. In an example, the depth sensor 230 can be configured to render a data set that can be used for clustering and object identification. For example, if the data indicate a relatively large continuous plane at a common depth, then an object can be identified at the common depth. Other techniques can similarly be used; para. [0052, 0057]).
Stein does not explicitly teach triangulation.
However, Adsumilli teaches wherein the first location is determined via triangulation (i.e. The visual object detection module 240 may determine a position for each visual object. For example, in a three-dimensional environment, the visual object detection module 240 may triangulate the positions of one or more visual objects based on parallax between two or more of the videos in which the visual objects appear. In some embodiments, the visual object detection module 240 may determine the position of a visual object based on depth maps corresponding to each of the cameras that captured the visual object. In some embodiments, the estimation of the visual object's position may be based on a combination of parallax and depth maps. In some embodiments, the change in position of each visual object may be tracked with a mean-shift and/or continuously adaptive mean shift (CAMshift) algorithm; para. [0056]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein, Younessian, and Zhou to include the feature of Adsumilli. One would have been motivated to make this modification because it allows for a precise determination of the visual’s object location by using multiple reference points.
Claim 14 is similar in scope to Claim 7 and is rejected under a similar rationale.
8. Claim 43 is rejected under 35 U.S.C. 103 as being unpatentable over Stein in view of Younessian, Zhou, and further in view of Yu et al. (U.S. Patent Application Pub. No. US 20040047464 A1).
Claim 43: Stein, Younessian, and Zhou teach the apparatus of claim 5. Stein does not explicitly teach to enhance the second audio stream by suppressing audio not associated with the first audio object via adaptive noise cancellation.
Yu teaches to enhance the second audio stream by suppressing audio not associated with the first audio object via adaptive noise cancellation (i.e. an adaptive noise canceling microphone system is provided, the adaptive noise canceling microphone system comprises two microphones being arranged at a predefined distance from each other, a signal forming system (SFS) for receiving a first and second input signal resulting from sounds received by the two microphones wherein an acoustical signal component in the first input signal is determined, wherein an acoustical signal component in the second input signal is determined, wherein the acoustical signal component in the first input signal is enhanced to generate a speech enhanced signal and wherein the acoustical signal component in the second input signal is suppressed to generate a speech nulled signal, and an adaptive noise cancellation filtering circuit for receiving the speech enhanced signal and the speech nulled signal, wherein the noise in the speech enhanced signal is cancelled using the speech nulled signal as reference, thereby generating an output filtered signal representing the desired signal; para. [0010]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Stein, Younessian, and Zhou to include the feature of Yu. One would have been motivated to make this modification because it ensures that the most relevant audio remains distinct and intelligible.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Poliakov et al. (Pub. No. US 10445938 B1), methods are presented for presentation of modified objects within a video stream. The systems and methods receive a set of images within a video stream and identify at least a portion of a face in a first subset of images.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAN TRAN whose telephone number is (303)297-4266. The examiner can normally be reached on Monday - Thursday - 8:00 am - 5:00 pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAN H TRAN/Primary Examiner, Art Unit 2141
Prosecution Insights