DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4, 10, 11, 12, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui).
Regarding claim 1, Kim teaches a virtual environment content management apparatus (Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion.) comprising: a communication unit configured to receive (Kim, paragraph 48, the communication device 110 may support the VDS device 100 to communicate with the ECUs mounted on the vehicle. The communication device 110 may include a transceiver that transmits and receives data (information) using a vehicle communication technology such as a controller area network (CAN), an Ethernet, and the like.) vehicle driving information and (Kim, paragraph 59, The processor 170 may be configured to receive driving information, for example, the RPM, the torque, a speed, and/or a throttle opening amount, transmitted from the electronic control unit (ECU) in the vehicle ) receive sound source usage information from a user terminal (The present specification discloses the sound source usage information may include metadata of a sound source reproduced on a terminal. Similarly, Kim teaches a communication between a virtual driving simulation (VDS) and electronic device (terminal or server) to receive sound from a server, which corresponds to sound source usage information. Kim, paragraph 49, the communication device 110 may support the VDS device 100 to perform wired communication and/or wireless communication with an external electronic device (e.g., a terminal, a server, and the like). For example, the communication device 110 may download a sound source through communication with a server that provides the sound source.);
a non-transitory memory configured to store computer-executable instructions(Kim, paragraph 40, non-transitory computer readable media on a computer readable medium containing executable program instructions ); and
one or more processors coupled with the non-transitory memory, the one or more processors configured to execute the computer-executable instructions to implement(Kim, paragraph 40, non-transitory computer readable media on a computer readable medium containing executable program instructions executed by a processor)
a second processing unit configured to generate a sound signal based on the image information(Kim, paragraph 58, the processor 170 may be configured to generate the 3D sound based on surrounding environment information. That is, the processor 170 may be configured to design the sound corresponding to the virtual image), the vehicle driving information and(Kim, paragraph 60, the processor 170 may be configured to generate a virtual sound from the engine sound generated by the ASD algorithm based on a head unit signal (a volume, a tone volume, and the like) and a performance signal (the driving information)),
a third processing unit configured to generate a vibration signal corresponding to the sound signal based on the image information and(Kim discloses generation of sound based on virtual image and further discloses generation of vibration based on sound. Kim, paragraph 55, The haptic controller 160 may be configured to generate vibration based on sound. Kim, paragraph 58, the processor 170 may be configured to generate the 3D sound based on surrounding environment information. That is, the processor 170 may be configured to design the sound corresponding to the virtual image )
a fourth processing unit configured to reconstruct the virtual environment content based on the sound signal and the vibration signal(Kim’s construction of virtual environment based on haptic controller that generates the vibration signal and sound-based tactile stimulus corresponds to the reconstruction of the virtual environment based on the sound signal and the vibration signal. Kim, paragraph 93, the processor 170 may be configured to construct a driving system test environment (the virtual environment) through the tuning of the haptic controller 160 and the VR device. The processor 170 may be configured to construct an environment for the sound-based tactile stimulus by examining a test mode, a road, the user, selection of a vehicle to be evaluated, a posture, and an evaluator optimal position setting screen.).
While Kim teaches about generating sound and vibration based on virtual image and vehicle driving information, it specifically fails to disclose a first processing unit configured to analyze image information of virtual environment content, an occupant detection information from a sensor unit of a vehicle; generate a sound signal based on the occupant detection information; vibration signal corresponding to the sound signal based on the image information and the occupant detection information; generate a sound signal based on the sound source usage information
However, Kim2, which is in the same analogous art and that teaches about generating a virtual sound based on vehicle environment data, discloses a first processing unit configured to analyze image information of virtual environment content(Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound based on the estimated driving environment); generate a sound signal based on the sound source usage information(Kim2 discloses processing different sounds such as animal sound to produce synthesized sound based on sound source analysis. Kim2, paragraph 75, the processing device 150 may be configured to perform sound quality synthesis of an animal sound, which passes through a sound source analysis, and a default sound for vehicle development using a formant filter. ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim with Kim2 to analyze image data obtained by camera to generate a virtual sound. By analyzing the image and generating sound, it is possible to adjust sound based on surrounding of the vehicle. For example, while driving on a country road the sound can be lowered compared to a busy downtown where the sound is increased. (Kim2, paragraph 53, he processing device 150 may be configured to generate a virtual sound (or an emotional sound, a zero to hundred sound, an acceleration sound, or the like) in conjunction with a driving environment (e.g., a country road, a downtown, the inside of tunnel, or the like), a vehicle speed, an RPM, accelerator pedal responsiveness, and/or the like in the zero to hundred condition).
The combination of Kim and Kim2 specifically fails to disclose an occupant detection information from a sensor unit of a vehicle; generate a sound signal based on the occupant detection information; vibration signal corresponding to the sound signal based on the image information and the occupant detection information.
However, Yasui, which is in the same analogous art and that teaches about an experience providing system discloses an occupant detection information from a sensor unit of a vehicle(The present specification discloses occupant detection information may include an occupant detection by determining a pressure signal generated by converting the pressure applied to the vehicle seat into an electrical signal, and detecting audio signal generated by collecting voices and humming sounds expressed by the occupant. Similarly, Yasui teaches passenger detection by different sensors, determining the pressure on a seat, and capturing humming sound and voice of a passenger to trigger an action. Yasui, paragraph 64, FIG. 2 is a diagram showing an example of a vehicle-mounted device including the passenger information detector 220. In the example in FIG. 2, the passenger information detector 220 includes an in-vehicle camera 220A, an in-vehicle microphone 220B, and a seating sensor 220C. The in-vehicle camera 220A images a face, for example, of the passenger and outputs the image. Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C. Yasui, paragraph 161, the local event determiner 520 acquires the sound data and the voice data, and in a case in which it is determined from the data that the passenger is performing an action of hamming or tapping a rhythm, the local event determiner 520 determines that an event that the passenger “shows its interest in the music” has occurred ); generate a sound signal based on the occupant detection information(Yasui discloses when detecting presence and utterance of an occupant, triggering an interaction content such as sound output synchronized (or coordinated) with actions of the virtual image. Yasui, paragraph 108, in a case in which a facial expression and the like of the passenger detected by the passenger information detector 220 of the vehicle 200 and/or a scene or the like outside the vehicle detected by the outside information detector 230 change, for example, the utterance generator 132A may generate original data such that an utterance of the virtual image output from the sound output section 244 dynamically changes, and also, the action generator 132C may generate original data such that actions of the virtual image generated by the virtual image generator 242 dynamically change, in accordance with the changes in facial expression and/or the scene. Yasui, paragraph 59, the content player 246 provides, to the passenger of the vehicle 200, content obtained by causing sound output from the sound output section 244 to be synchronized (or coordinated) with actions of the virtual image that is caused to appear by the virtual image generator 242. That is, the content player 246 causes the sound (for example, a music) output from the sound output section 244 to be synchronized with display (for example, actions of the virtual image) output from the virtual image generator 242); Vibration signal corresponding to the sound signal based on the image information and the occupant detection information (Kim discloses the generation of vibration signal based on the image information and Yasui teaches detection of occupants to trigger interactions such sound generation in coordination with virtual image. While Kim does not specifically disclose the generation of vibration based on the detection of an occupant, a person of ordinary skill in the art would be able to modify Kim’s vibration producing mechanism with Yasui to generate vibration when occupant is detected. The person of ordinary skill in the art will be motivated to combine Kim and Yasui to generate vibration based confirmation that a person is present in the vehicle, as it would be unnecessary to generate vibration if a person is not present in the vehicle while the vehicle is turned on. Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C. Kim, paragraph 3, an input device that receives a movement of the user as an input, and a control device that changes the virtual space by reflecting the movement of the user input through the input device and outputs the changed virtual space on the display device. A sensation of immersion technology must be added between a virtual image and an interaction to implement a virtual environment in such a virtual reality system.).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Kim2 with Yasui to generate a sound signal based on the occupant detection information. By generating sound signal when occupant is detected, it is possible to establishes communication between the passenger of the vehicle and the virtual image by creating an interactive dialogue while only generating sound when occupant is detected with motivation to generate sound based on user contextual information. For example, the sound generated for an adult maybe different than a sound generation for child. Moreover, it would be unnecessary to produce sound based on the virtual image if a person is not present in the vehicle while the vehicle is turned on. Furthermore, it is possible to dynamically change sound corresponding to detected user’s facial expression instead of static sound. ( Kim, paragraph 94, The utterance generator 132A generates original data of an utterance of the virtual image on the basis of the utterance of the passenger of the vehicle 200 detected by the passenger information detector 220 of the vehicle 200 and establishes communication between the passenger of the vehicle 200 and the virtual image. Kim, paragraph 108, in a case in which a facial expression and the like of the passenger detected by the passenger information detector 220 of the vehicle 200 and/or a scene or the like outside the vehicle detected by the outside information detector 230 change, for example, the utterance generator 132A may generate original data such that an utterance of the virtual image output from the sound output section 244 dynamically changes, and also, the action generator 132C may generate original data such that actions of the virtual image generated by the virtual image generator 242 dynamically change, in accordance with the changes in facial expression and/or the scene).
Regarding claim 2, the combination of Kim, Kim2, and Yasui teaches the combination of The virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C ), wherein the second processing unit is configured to set the sound signal as one of an entertainment type(Yasui, paragraph 58, the sound output section 244 outputs sound for entertaining the passenger of the vehicle 200 ), a sandbox type, and a professional type based on the sound source usage information.
Regarding claim 4, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the second processing unit is configured to set voice parameters corresponding to image parameters constituting the image information (Yasui, paragraph 104, The singing voice generator 132B generates original data of a singing voice of the virtual image output from the sound output section 244 of the vehicle 200. Yasui, paragraph 90 , The original data generator 132 generates original data of content that is obtained by causing the sound (for example, singing, an utterance, and the like of the virtual image) output from the sound output section 244 of the vehicle 200 to be coordinated with actions of the virtual image generated by the virtual image generator 242 of the vehicle 200, for the passenger of the vehicle 200) and generate the sound signal based on the voice parameters (Yasui, paragraph 93 , the interactive voice response function is a function of responding to an utterance of the passenger of the vehicle 200 using sound recorded in advance or sound that is dynamically generated).
Regarding claim 10, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),wherein the fourth processing unit is configured to generate a reproduction signal obtained by synchronizing the sound signal and the vibration signal with the image information of the virtual environment content(Kim’s construction of virtual environment based on haptic controller that generates the vibration signal and sound-based tactile stimulus (that is based on virtual image) corresponds to the reproduction signal obtained by synchronizing the sound signal and the vibration signal with the image information of the virtual environment content. Kim, paragraph 93, the processor 170 may be configured to construct a driving system test environment (the virtual environment) through the tuning of the haptic controller 160 and the VR device. The processor 170 may be configured to construct an environment for the sound-based tactile stimulus by examining a test mode, a road, the user, selection of a vehicle to be evaluated, a posture, and an evaluator optimal position setting screen. Kim, paragraph 53, the processor 170 may design a 3D sound matching the virtual environment. In other words, the processor 170 may generate a sound (a 3D sound) corresponding to a virtual image).
Regarding claim 11, Kim teaches a virtual environment content management method(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion) comprising:
receiving, by a communication unit(Kim, paragraph 48, the communication device 110 may support the VDS device 100 to communicate with the ECUs mounted on the vehicle. The communication device 110 may include a transceiver that transmits and receives data (information) using a vehicle communication technology such as a controller area network (CAN), an Ethernet, and the like), vehicle driving information and (Kim, paragraph 59, The processor 170 may be configured to receive driving information, for example, the RPM, the torque, a speed, and/or a throttle opening amount, transmitted from the electronic control unit (ECU) in the vehicle) sound source usage information from a user terminal(The present specification discloses the sound source usage information may include metadata of a sound source reproduced on a terminal. Similarly, Kim teaches a communication between a virtual driving simulation (VDS) and electronic device (terminal or server) to receive sound from a server, which corresponds to sound source usage information. Kim, paragraph 49, the communication device 110 may support the VDS device 100 to perform wired communication and/or wireless communication with an external electronic device (e.g., a terminal, a server, and the like). For example, the communication device 110 may download a sound source through communication with a server that provides the sound source);
generating, by the processor, a sound signal based on the image information(Kim, paragraph 58, the processor 170 may be configured to generate the 3D sound based on surrounding environment information. That is, the processor 170 may be configured to design the sound corresponding to the virtual image), the vehicle driving information, and(Kim, paragraph 60, the processor 170 may be configured to generate a virtual sound from the engine sound generated by the ASD algorithm based on a head unit signal (a volume, a tone volume, and the like) and a performance signal (the driving information));
generating, by the processor, a vibration signal corresponding to the sound signal based on the image information and(Kim discloses generation of sound based on virtual image and further discloses generation of vibration based on sound. Kim, paragraph 55, The haptic controller 160 may be configured to generate vibration based on sound. Kim, paragraph 58, the processor 170 may be configured to generate the 3D sound based on surrounding environment information. That is, the processor 170 may be configured to design the sound corresponding to the virtual image) ; and
reconstructing, by the processor, the virtual environment content based on the sound signal and the vibration signal(Kim’s construction of virtual environment based on haptic controller that generates the vibration signal and sound-based tactile stimulus corresponds to the reconstruction of the virtual environment based on the sound signal and the vibration signal. Kim, paragraph 93, the processor 170 may be configured to construct a driving system test environment (the virtual environment) through the tuning of the haptic controller 160 and the VR device. The processor 170 may be configured to construct an environment for the sound-based tactile stimulus by examining a test mode, a road, the user, selection of a vehicle to be evaluated, a posture, and an evaluator optimal position setting screen).
While Kim teaches about generating sound and vibration based on virtual image and vehicle driving information, it specifically fails to disclose occupant detection information from a sensor unit of a vehicle and analyzing, by a processor, image information of virtual environment content; generate a sound signal based on the occupant detection information; vibration signal corresponding to the sound signal based on the image information and the occupant detection information; generate a sound signal based on the sound source usage information
However, Kim2, which is in the same analogous art and that teaches about generating a virtual sound based on vehicle environment data, discloses analyzing, by a processor, image information of virtual environment content(Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound based on the estimated driving environment); generate a sound signal based on the sound source usage information(Kim2 discloses processing different sounds such as animal sound to produce synthesized sound based on sound source analysis. Kim2, paragraph 75, the processing device 150 may be configured to perform sound quality synthesis of an animal sound, which passes through a sound source analysis, and a default sound for vehicle development using a formant filter).
The combination of Kim and Kim2 specifically fails to disclose an occupant detection information from a sensor unit of a vehicle(The present specification discloses occupant detection information may include an occupant detection by determining a pressure signal generated by converting the pressure applied to the vehicle seat into an electrical signal, and detecting audio signal generated by collecting voices and humming sounds expressed by the occupant. Similarly, Yasui teaches passenger detection by different sensors, determining the pressure on a seat, and capturing humming sound and voice of a passenger to trigger an action. Yasui, paragraph 64, FIG. 2 is a diagram showing an example of a vehicle-mounted device including the passenger information detector 220. In the example in FIG. 2, the passenger information detector 220 includes an in-vehicle camera 220A, an in-vehicle microphone 220B, and a seating sensor 220C. The in-vehicle camera 220A images a face, for example, of the passenger and outputs the image. Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C. Yasui, paragraph 161, the local event determiner 520 acquires the sound data and the voice data, and in a case in which it is determined from the data that the passenger is performing an action of hamming or tapping a rhythm, the local event determiner 520 determines that an event that the passenger “shows its interest in the music” has occurred ), generate a sound signal based on the occupant detection information(Yasui discloses when detecting presence and utterance of an occupant, triggering an interaction content such as sound output synchronized (or coordinated) with actions of the virtual image. Yasui, paragraph 108, in a case in which a facial expression and the like of the passenger detected by the passenger information detector 220 of the vehicle 200 and/or a scene or the like outside the vehicle detected by the outside information detector 230 change, for example, the utterance generator 132A may generate original data such that an utterance of the virtual image output from the sound output section 244 dynamically changes, and also, the action generator 132C may generate original data such that actions of the virtual image generated by the virtual image generator 242 dynamically change, in accordance with the changes in facial expression and/or the scene. Yasui, paragraph 59, the content player 246 provides, to the passenger of the vehicle 200, content obtained by causing sound output from the sound output section 244 to be synchronized (or coordinated) with actions of the virtual image that is caused to appear by the virtual image generator 242. That is, the content player 246 causes the sound (for example, a music) output from the sound output section 244 to be synchronized with display (for example, actions of the virtual image) output from the virtual image generator 242); vibration signal corresponding to the sound signal based on the image information and the occupant detection information(Kim discloses the generation of vibration signal based on the image information and Yasui teaches detection of occupants to trigger interactions such sound generation in coordination with virtual image. While Kim does not specifically disclose the generation of vibration based on the detection of an occupant, a person of ordinary skill in the art would be able to modify Kim’s vibration producing mechanism with Yasui to generate vibration when occupant is detected. The person of ordinary skill in the art will be motivated to combine Kim and Yasui to generate vibration based confirmation that a person is present in the vehicle, as it would be unnecessary to generate vibration if a person is not present in the vehicle while the vehicle is turned on. Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C. Kim, paragraph 3, an input device that receives a movement of the user as an input, and a control device that changes the virtual space by reflecting the movement of the user input through the input device and outputs the changed virtual space on the display device. A sensation of immersion technology must be added between a virtual image and an interaction to implement a virtual environment in such a virtual reality system. ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Kim2 with Yasui to generate a sound signal based on the occupant detection information. By generating sound signal when occupant is detected, it is possible to establishes communication between the passenger of the vehicle and the virtual image by creating an interactive dialogue while only generating sound when occupant is detected with motivation to generate sound based on user contextual information. For example, the sound generated for an adult maybe different than a sound generation for child. Moreover, it would be unnecessary to produce sound based on the virtual image if a person is not present in the vehicle while the vehicle is turned on. Furthermore, it is possible to dynamically change sound corresponding to detected user’s facial expression instead of static sound. ( Kim, paragraph 94, The utterance generator 132A generates original data of an utterance of the virtual image on the basis of the utterance of the passenger of the vehicle 200 detected by the passenger information detector 220 of the vehicle 200 and establishes communication between the passenger of the vehicle 200 and the virtual image. Kim, paragraph 108, in a case in which a facial expression and the like of the passenger detected by the passenger information detector 220 of the vehicle 200 and/or a scene or the like outside the vehicle detected by the outside information detector 230 change, for example, the utterance generator 132A may generate original data such that an utterance of the virtual image output from the sound output section 244 dynamically changes, and also, the action generator 132C may generate original data such that actions of the virtual image generated by the virtual image generator 242 dynamically change, in accordance with the changes in facial expression and/or the scene).
Regarding claim 12, the combination of Kim, Kim2, and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the generating of the sound signal includes setting, by the processor, the sound signal as one of an entertainment type(Yasui, paragraph 58, the sound output section 244 outputs sound for entertaining the passenger of the vehicle 200 ), a sandbox type, and a professional type based on the sound source usage information.
Regarding claim 14, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the generating of the sound signal includes:
setting, by the processor, voice parameters corresponding to image parameters constituting the image information(Yasui, paragraph 104, The singing voice generator 132B generates original data of a singing voice of the virtual image output from the sound output section 244 of the vehicle 200. Yasui, paragraph 90 , The original data generator 132 generates original data of content that is obtained by causing the sound (for example, singing, an utterance, and the like of the virtual image) output from the sound output section 244 of the vehicle 200 to be coordinated with actions of the virtual image generated by the virtual image generator 242 of the vehicle 200, for the passenger of the vehicle 200); and
generating, by the processor, the sound signal based on the voice parameters(Yasui, paragraph 93 , the interactive voice response function is a function of responding to an utterance of the passenger of the vehicle 200 using sound recorded in advance or sound that is dynamically generated).
Regarding claim 20, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the reconstructing of the virtual environment content includes generating, by the processor, a reproduction signal obtained by synchronizing the sound signal and the vibration signal with the image information of the virtual environment content(Kim’s construction of virtual environment based on haptic controller that generates the vibration signal and sound-based tactile stimulus (that is based on virtual image) corresponds to the reproduction signal obtained by synchronizing the sound signal and the vibration signal with the image information of the virtual environment content. Kim, paragraph 93, the processor 170 may be configured to construct a driving system test environment (the virtual environment) through the tuning of the haptic controller 160 and the VR device. The processor 170 may be configured to construct an environment for the sound-based tactile stimulus by examining a test mode, a road, the user, selection of a vehicle to be evaluated, a posture, and an evaluator optimal position setting screen. Kim, paragraph 53, the processor 170 may design a 3D sound matching the virtual environment. In other words, the processor 170 may generate a sound (a 3D sound) corresponding to a virtual image).
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui) in further view of Youn (US 20200130674 A1)(hereinafter Youn).
Regarding claim 3, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 2(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim, Kim2 and Yasui specifically fails to disclose a system wherein the second processing unit is configured to extract emotional adjectives based on the sound source usage information and quantify contribution of the emotional adjectives to set the sound signal as one of an entertainment type, a sandbox type, and a professional type.
However, Youn, which is in the same analogous art and that teaches about controlling a size and a direction of sound output through a vehicle’s speaker, discloses a system wherein the second processing unit is configured to extract emotional adjectives based on the sound source usage information (Youn, paragraph 69, The tag information manager 244 may extract the controlled information by the speaker 260 to generate the emotion tag information. Youn, paragraph 94, The controller 240 may generate the emotion tag information related to the output sound source by using the passenger's emotional information for the output sound source, and the controlled output size information of the speaker 260 and/or the controlled output direction of the speaker 260 (670). ) and quantify contribution of the emotional adjectives to set the sound signal as one of an entertainment type(Youn discloses outputting different sounds such as music reproduction function, a radio function, and a video function which correspond to entertainment type sound signals. Youn, paragraph 52, the speaker 260 provided in the vehicle 1 may output sounds related to the performance of various devices in the vehicle 1. For example, the speaker 260 may output sound required to perform a music reproduction function, a radio function, a video function ), a sandbox type, and a professional type.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim with Yasui with Youn to extract emotion tag information based on sound source. By extracting the emotion tag, it is possible to appropriately distribute the sound output through the speaker in the vehicle and improve the auditory comfort of the passenger.( Youn, paragraph 95, obtain the passenger's emotional information with respect to content output through the speaker in the vehicle and automatically control a size and an output direction of a speaker output according to the passenger's emotional information, and a method of controlling the vehicle. Therefore, it is possible to accurately grasp the passenger's emotion, appropriately distribute the sound output through the speaker in the vehicle and improve the auditory comfort of the passenger).
Regarding claim 13, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 12(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim, Kim2 and Yasui specifically fails to disclose a method wherein the generating of the sound signal includes:
extracting, by the processor, emotional adjectives based on the sound source usage information; and
quantifying, by the processor, contribution of the emotional adjectives to set the sound signal as one of an entertainment type, a sandbox type, and a professional type.
However, Youn, which is in the same analogous art and that teaches about controlling a size and a direction of sound output through a vehicle’s speaker, discloses a method wherein the generating of the sound signal includes:
extracting, by the processor, emotional adjectives based on the sound source usage information(Youn, paragraph 69, The tag information manager 244 may extract the controlled information by the speaker 260 to generate the emotion tag information. Youn, paragraph 94, The controller 240 may generate the emotion tag information related to the output sound source by using the passenger's emotional information for the output sound source, and the controlled output size information of the speaker 260 and/or the controlled output direction of the speaker 260 (670).); and
quantifying, by the processor, contribution of the emotional adjectives to set the sound signal as one of an entertainment type(Youn discloses outputting different sounds such as music reproduction function, a radio function, and a video function which correspond to entertainment type sound signals. Youn, paragraph 52, the speaker 260 provided in the vehicle 1 may output sounds related to the performance of various devices in the vehicle 1. For example, the speaker 260 may output sound required to perform a music reproduction function, a radio function, a video function), a sandbox type, and a professional type.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim with Yasui with Youn to extract emotion tag information based on sound source. By extracting the emotion tag, it is possible to appropriately distribute the sound output through the speaker in the vehicle and improve the auditory comfort of the passenger.( Youn, paragraph 95, obtain the passenger's emotional information with respect to content output through the speaker in the vehicle and automatically control a size and an output direction of a speaker output according to the passenger's emotional information, and a method of controlling the vehicle. Therefore, it is possible to accurately grasp the passenger's emotion, appropriately distribute the sound output through the speaker in the vehicle and improve the auditory comfort of the passenger).
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui) in further view of Nishihara (JP 2010122566 A)(hereinafter Nishihara) in further view of Liao (CN 110134122A)(hereinafter Liao).
Regarding claim 5, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 4(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim and Yasui specifically fails to disclose a system wherein the second processing unit is configured to extract the image parameters including impact intensity, color, brightness, and set the image parameters to the voice parameters including volume, timbre, pitch,
However, Nishihara, which is in the same analogous art and that teaches about a color/tone conversion device discloses a system wherein the second processing unit is configured to extract the image parameters including impact intensity (According to the specification, the impact intensity may refer to the intensity of a specific event, background, movement, and the like within the image. Similarly, Nishihara discloses sound pitch and volume can be changed based on movement of object retrieved from an image. Nishihara, paragraph 50, images projected without intention to be converted into sound are projected onto the wall surface 3 by the projector 2, or an actual outdoor scene or indoor scene can be used without using the projector 2 or the wall surface 3…. the pitch, volume, etc. can be changed according to the movement of the room, etc. In the case of an indoor scene, the pitch, volume, etc., depending on the movement of the person, lighting / extinguishing of lighting equipment, use of fire in the kitchen, etc. Can be changed ), color, brightness, and ( Nishihara, paragraph 6, a color separation unit that separates incident visible light into light of a plurality of colors, and a signal level corresponding to the amount of light received by the light separated by the color separation unit ) set the image parameters to the voice parameters including volume, timbre, pitch, and(Nishihara discloses controlling the sound’s tone color(timbre) based on blue component of an image. Nishihara, paragraph 50, The pitch, volume, etc. can be changed according to the movement of the room, etc. In the case of an indoor scene, the pitch, volume, etc., depending on the movement of the person, lighting / extinguishing of lighting equipment, use of fire in the kitchen, etc. Can be changed. Nishihara, paragraph, the sound information is sent continuously in this way, the brightness of the green component (volume) and the brightness of the blue component (tone color) while the brightness (pitch) of the red component is kept constant. ) Can be changed to add sound effects such as transitions (envelopes) of volume changes over time, vibrato, and echoes).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 and Yasui with Nishihara to set image parameters to a voice parameters such as volume and pitch. By dynamically adjusting voice/sound parameters based on image data, it is possible to improve user experience by continuously changing the pitch of sound in accordance with user’s unique atmosphere. (Nishihara, paragraph 13, it is possible to produce sounds with a pitch that is more finely divided than semitones, and to change the pitch continuously, so that a sound with a unique atmosphere can be created. Nishihara, paragraph 16, the acoustic signal generated by the acoustic signal generating means can change not only the pitch and volume but also the timbre, the generated sound is prevented from becoming monotonous and rich).
The combination of Kim, Kim2, Yasui, and Nishihara specifically fails to determining contact time, and set the image parameters to the voice parameters including duration.
However, Liao, which is in the same analogous art and that teaches about high-spectrum imaging, discloses determining contact time(The present specification discloses that contact time can be the time that vehicles are in contact with each other in a collision accident. Similarly, Liao discloses determining the collision time of a vehicle. Liao, paragraph 113, the operation processor 22 also calculates the vehicle collide with the carriage time reaches the preset threshold value according to the current vehicle state of the vehicle of the current vehicle state and the carriage, the loudspeaker audio intelligent loudspeaker audio database module 25 corresponding to the output to the execution controller 23, a vehicle horn of execution controller 23 further controls the vehicle emits horn sound corresponding to the corresponding loudspeaker audio.), and set the image parameters to the voice parameters including duration(Liao further discloses controlling the length of sound from loudspeaker based on collision time, which corresponds to the duration of the voice parameter. Liao, paragraph 31, a vehicle horn further proposal of the invention, the execution controller controls the vehicle according to the corresponding speaker audio to produce corresponding volume, the loudspeaker vocal pitch and length. Liao, paragraph 114, the current vehicle state with the vehicle acquired by the current vehicle state, calculating the collision time T1 of the own vehicle and a leading, assuming T1 satisfies 0.5 seconds ≤ T1 < 1 seconds, can be volume is 90 db, the pitch is 800Hz, the horn sound duration is 1 seconds, calculating the vehicle collision time T2 with sides of the car satisfies 1 seconds ≤ T2 < 1.5 seconds, the selected volume is 85 decibel, the pitch is 900Hz, the horn sound duration is 1 seconds ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 Yasui, and Nishihara with Liao to determine the duration of sound output based on collision time. By determining the duration, it is possible to improve the recognition of the speaker audio during an accident and increase the safety during automatic driving. (Liao, paragraph 115, automatically adjust the vehicle horn volume, pitch and length according to the automatic driving peripheral environment factor, so as to effectively improve the resolution of speaker audio and safety of automatic driving ).
Regarding claim 15, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 14(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim,Kim2, and Yasui specifically fails to disclose a method wherein the generating of the sound signal includes:
extracting, by the processor, the image parameters including impact intensity, color, brightness, and contact time; and
setting, by the processor, the image parameters to the voice parameters including volume, timbre, pitch, and duration.
However, Nishihara, which is in the same analogous art and that teaches about a color/tone conversion device discloses a method wherein the generating of the sound signal includes:
extracting, by the processor, the image parameters including impact intensity(According to the specification, the impact intensity may refer to the intensity of a specific event, background, movement, and the like within the image. Similarly, Nishihara discloses sound pitch and volume can be changed based on movement of object retrieved from an image. Nishihara, paragraph 50, images projected without intention to be converted into sound are projected onto the wall surface 3 by the projector 2, or an actual outdoor scene or indoor scene can be used without using the projector 2 or the wall surface 3…. the pitch, volume, etc. can be changed according to the movement of the room, etc. In the case of an indoor scene, the pitch, volume, etc., depending on the movement of the person, lighting / extinguishing of lighting equipment, use of fire in the kitchen, etc. Can be changed ), color, brightness, and(Nishihara, paragraph 6, a color separation unit that separates incident visible light into light of a plurality of colors, and a signal level corresponding to the amount of light received by the light separated by the color separation unit);
setting, by the processor, the image parameters to the voice parameters including volume, timbre, pitch, and(Nishihara discloses controlling the sound’s tone color(timbre) based on blue component of an image. Nishihara, paragraph 50, The pitch, volume, etc. can be changed according to the movement of the room, etc. In the case of an indoor scene, the pitch, volume, etc., depending on the movement of the person, lighting / extinguishing of lighting equipment, use of fire in the kitchen, etc. Can be changed. Nishihara, paragraph, the sound information is sent continuously in this way, the brightness of the green component (volume) and the brightness of the blue component (tone color) while the brightness (pitch) of the red component is kept constant. ) Can be changed to add sound effects such as transitions (envelopes) of volume changes over time, vibrato, and echoes).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 and Yasui with Nishihara to set image parameters to a voice parameters such as volume and pitch. By dynamically adjusting voice/sound parameters based on image data, it is possible to improve user experience by continuously changing the pitch of sound in accordance with user’s unique atmosphere. (Nishihara, paragraph 13, it is possible to produce sounds with a pitch that is more finely divided than semitones, and to change the pitch continuously, so that a sound with a unique atmosphere can be created. Nishihara, paragraph 16, the acoustic signal generated by the acoustic signal generating means can change not only the pitch and volume but also the timbre, the generated sound is prevented from becoming monotonous and rich).
The combination of Kim, Kim2, Yasui, and Nishihara specifically fails to determining contact time, and set the image parameters to the voice parameters including duration.
However, Liao, which is in the same analogous art and that teaches about high-spectrum imaging, discloses determining contact time(The present specification discloses that contact time can be the time that vehicles are in contact with each other in a collision accident. Similarly, Liao discloses determining the collision time of a vehicle. Liao, paragraph 113, the operation processor 22 also calculates the vehicle collide with the carriage time reaches the preset threshold value according to the current vehicle state of the vehicle of the current vehicle state and the carriage, the loudspeaker audio intelligent loudspeaker audio database module 25 corresponding to the output to the execution controller 23, a vehicle horn of execution controller 23 further controls the vehicle emits horn sound corresponding to the corresponding loudspeaker audio.), and set the image parameters to the voice parameters including duration(Liao further discloses controlling the length of sound from loudspeaker based on collision time, which corresponds to the duration of the voice parameter. Liao, paragraph 31, a vehicle horn further proposal of the invention, the execution controller controls the vehicle according to the corresponding speaker audio to produce corresponding volume, the loudspeaker vocal pitch and length. Liao, paragraph 114, the current vehicle state with the vehicle acquired by the current vehicle state, calculating the collision time T1 of the own vehicle and a leading, assuming T1 satisfies 0.5 seconds ≤ T1 < 1 seconds, can be volume is 90 db, the pitch is 800Hz, the horn sound duration is 1 seconds, calculating the vehicle collision time T2 with sides of the car satisfies 1 seconds ≤ T2 < 1.5 seconds, the selected volume is 85 decibel, the pitch is 900Hz, the horn sound duration is 1 seconds ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 Yasui, and Nishihara with Liao to determine the duration of sound output based on collision time. By determining the duration, it is possible to improve the recognition of the speaker audio during an accident and increase the safety during automatic driving. (Liao, paragraph 115, automatically adjust the vehicle horn volume, pitch and length according to the automatic driving peripheral environment factor, so as to effectively improve the resolution of speaker audio and safety of automatic driving ).
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui) in further view of Gao (CN 118400660 A)(hereinafter Gao).
Regarding claim 6, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the second processing unit is configured to adjust volume, timbre (Kim2, paragraph 49, The sound playback device may be configured to adjust and play volume, a tone (or sound quality), a sound image, and the like of the sound under an instruction of the processing device 150. ) of the sound signal based on an accelerator pedal response and a vehicle speed(Kim2, paragraph 20, the generating of the virtual sound may comprise implementing the virtual sound in three steps based on a vehicle speed and accelerator pedal responsiveness. Kim2, paragraph 42, the apparatus 100 for generating the virtual sound may be configured to design a virtual sound based on a hearing experience of a user and may be configured to personalize the virtual sound by means of tone adjustment and accelerator pedal responsiveness adjustment.).
The combination of Kim, Kim2, and Yasui specifically fails to disclose a device wherein the second processing unit is configured to adjust sound spatial information of the sound signal based on an accelerator pedal response and a vehicle speed.
However, Gao, which is in the same analogous art and that teaches about generating sound wave in vehicle discloses a device to adjust sound spatial information of the sound signal based on an accelerator pedal response and a vehicle speed(Gao discloses adjusting the surround sound algorithm based on accelerator pedal and vehicle speed, which corresponds to the sound spatial information. Gao, paragraph 17, through the self-adaptive AVAS noise reduction algorithm integrated in the ADSP or QNX operating system of the SOC of the cabin domain control and the constructed panoramic sound stereo and/or sound wave algorithm of surround sound, by obtaining accelerator pedal, vehicle speed, gear, the control signal performs the playback and calibration of the panoramic sound wave, and plays the sound wave audio signal of the 3D panoramic sound through the sound equipment).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 and Yasui with Gao to adjust 3D(surround) sound and volume based on accelerator pedal response and a vehicle speed. By adjusting the volume and surround sound based on accelerator pedal response and a vehicle speed, it is possible to improve hearing experience of the 3D sound for the driver based on the state of the vehicle. Furthermore, it is possible to generate personalized sounds that are emitted for different car engines during high or low speed vehicle drive.(Gao, paragraph 17, The invention uses the noise reduction algorithm combined with the algorithm to perform playback and calibration on the panoramic sound wave and then output and play, which improves the signal-to-noise ratio of the 3D panoramic sound wave, improves the quality of the audio signal obtained by the sound device, and makes the 3D panoramic sound wave to perform self-adapting change along with the change of the vehicle state, It improves the feeling of the 3D panoramic sound wave impact in the vehicle. Gao, paragraph 35, the graphical sound action track are processed to generate the personalized sound emitted by the oil vehicle engine or the roadster engine or the exhaust pipe. when the vehicle is in the acceleration and deceleration state, the 3D panoramic sound wave scene in the vehicle is changed along with the change of the vehicle speed, for example, the 3D panoramic sound wave sound in the vehicle is gradually increased when the vehicle is accelerated, the 3D panoramic sound wave sound in the vehicle is gradually reduced when the vehicle is decelerated ).
Regarding claim 16, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the generating of the sound signal includes adjusting, by the processor, volume, timbre and(Kim2, paragraph 49, The sound playback device may be configured to adjust and play volume, a tone (or sound quality), a sound image, and the like of the sound under an instruction of the processing device 150.), based on an accelerator pedal response and a vehicle speed(Kim2, paragraph 20, the generating of the virtual sound may comprise implementing the virtual sound in three steps based on a vehicle speed and accelerator pedal responsiveness. Kim2, paragraph 42, the apparatus 100 for generating the virtual sound may be configured to design a virtual sound based on a hearing experience of a user and may be configured to personalize the virtual sound by means of tone adjustment and accelerator pedal responsiveness adjustment ).
The combination of Kim, Kim2, and Yasui specifically fails to disclose a device wherein the second processing unit is configured to adjust sound spatial information of the sound signal based on an accelerator pedal response and a vehicle speed.
However, Gao, which is in the same analogous art and that teaches about generating sound wave in vehicle discloses a device to adjust sound spatial information of the sound signal based on an accelerator pedal response and a vehicle speed(Gao discloses adjusting the surround sound algorithm based on accelerator pedal and vehicle speed, which corresponds to the sound spatial information. Gao, paragraph 17, through the self-adaptive AVAS noise reduction algorithm integrated in the ADSP or QNX operating system of the SOC of the cabin domain control and the constructed panoramic sound stereo and/or sound wave algorithm of surround sound, by obtaining accelerator pedal, vehicle speed, gear, the control signal performs the playback and calibration of the panoramic sound wave, and plays the sound wave audio signal of the 3D panoramic sound through the sound equipment).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim, Kim2 and Yasui with Gao to adjust 3D(surround) sound and volume based on accelerator pedal response and a vehicle speed. By adjusting the volume and surround sound based on accelerator pedal response and a vehicle speed, it is possible to improve hearing experience of the 3D sound for the driver based on the state of the vehicle. Furthermore, it is possible to generate personalized sounds that are emitted for different car engines during high or low speed vehicle drive.(Gao, paragraph 17, The invention uses the noise reduction algorithm combined with the algorithm to perform playback and calibration on the panoramic sound wave and then output and play, which improves the signal-to-noise ratio of the 3D panoramic sound wave, improves the quality of the audio signal obtained by the sound device, and makes the 3D panoramic sound wave to perform self-adapting change along with the change of the vehicle state, It improves the feeling of the 3D panoramic sound wave impact in the vehicle. Gao, paragraph 35, the graphical sound action track are processed to generate the personalized sound emitted by the oil vehicle engine or the roadster engine or the exhaust pipe. when the vehicle is in the acceleration and deceleration state, the 3D panoramic sound wave scene in the vehicle is changed along with the change of the vehicle speed, for example, the 3D panoramic sound wave sound in the vehicle is gradually increased when the vehicle is accelerated, the 3D panoramic sound wave sound in the vehicle is gradually reduced when the vehicle is decelerated ).
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui) in further view of Hughes (US 20120306631 A1) (hereinafter Hughes).
Regarding claim 7, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim and Yasui specifically fails to disclose a system wherein the third processing unit is configured to set a pattern of the vibration signal based on a background sound and a sound effect of the sound signal.
However, Hughes, which is in the same analogous art and that teaches about haptic communication in an extended reality environment discloses a system wherein the third processing unit is configured to set a pattern of the vibration signal based on a background sound (Hughes, paragraph 10, ambient noise can be detected by the mobile device, e.g., fire alarms, cars, emergency vehicles, car horns, screams, dog barking, music, environmental noise, phone ringing, knock at the door, etc. In response to detecting the ambient noise, a unique vibration pattern can be actuated by a haptic mechanism within a mobile device to augment the auditory information with haptic feedback ) and a sound effect of the sound signal(Hughes, paragraph 44, Utilizing audio subsystem 308 and haptic subsystem 314, although the user might have a hearing impairment relating to those higher frequencies, the user may still receive the information contained in those higher frequency ranges. For example, haptic subsystem 314 can convert the lower frequencies, e.g., sound effects and explosions, into vibration patterns and audio subsystem 310 can shift the higher frequencies that the user cannot hear into a lower range of frequencies which the user can hear. Hughes, paragraph 45, haptic subsystem 314 may be utilized to convert all or a portion of the sound effects track to haptic data and a corresponding vibration pattern. ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Yasui with Hughes to set a pattern of the vibration based on sound effect and ambient sound. By adjusting vibration pattern based on the sound effect, it is possible to improve the experience of a user with hearing impairment. ( Hughes, paragraph 44, a user of mobile device 300 may be watching a movie rich in high audible frequencies. In addition, the user may possess a hearing impairment that renders the user partially or completely deaf to a range of higher frequencies corresponding to talking or dialog in the movie. Utilizing audio subsystem 308 and haptic subsystem 314, although the user might have a hearing impairment relating to those higher frequencies, the user may still receive the information contained in those higher frequency ranges. For example, haptic subsystem 314 can convert the lower frequencies, e.g., sound effects and explosions, into vibration patterns and audio subsystem 310 can shift the higher frequencies that the user cannot hear into a lower range of frequencies which the user can hear. )
Regarding claim 17, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim and Yasui specifically fails to disclose a method wherein the generating of the sound signal includes setting, by the processor, a pattern of the vibration signal based on a background sound and a sound effect of the sound signal.
However, Hughes, which is in the same analogous art and that teaches about haptic communication in an extended reality environment discloses a method wherein the generating of the sound signal includes setting, by the processor, a pattern of the vibration signal based on a background sound(Hughes, paragraph 10, ambient noise can be detected by the mobile device, e.g., fire alarms, cars, emergency vehicles, car horns, screams, dog barking, music, environmental noise, phone ringing, knock at the door, etc. In response to detecting the ambient noise, a unique vibration pattern can be actuated by a haptic mechanism within a mobile device to augment the auditory information with haptic feedback) and a sound effect of the sound signal(Hughes, paragraph 44, Utilizing audio subsystem 308 and haptic subsystem 314, although the user might have a hearing impairment relating to those higher frequencies, the user may still receive the information contained in those higher frequency ranges. For example, haptic subsystem 314 can convert the lower frequencies, e.g., sound effects and explosions, into vibration patterns and audio subsystem 310 can shift the higher frequencies that the user cannot hear into a lower range of frequencies which the user can hear. Hughes, paragraph 45, haptic subsystem 314 may be utilized to convert all or a portion of the sound effects track to haptic data and a corresponding vibration pattern.).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Yasui with Hughes to set a pattern of the vibration based on sound effect and ambient sound. By adjusting vibration pattern based on the sound effect, it is possible to improve the experience of a user with hearing impairment. ( Hughes, paragraph 44, a user of mobile device 300 may be watching a movie rich in high audible frequencies. In addition, the user may possess a hearing impairment that renders the user partially or completely deaf to a range of higher frequencies corresponding to talking or dialog in the movie. Utilizing audio subsystem 308 and haptic subsystem 314, although the user might have a hearing impairment relating to those higher frequencies, the user may still receive the information contained in those higher frequency ranges. For example, haptic subsystem 314 can convert the lower frequencies, e.g., sound effects and explosions, into vibration patterns and audio subsystem 310 can shift the higher frequencies that the user cannot hear into a lower range of frequencies which the user can hear).
Claims 8, 9, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 20230186781 A1)(hereinafter Kim) in view of Kim (US 20230294662 A1) (hereinafter Kim2) in further view of Yasui (US 20210150772 A1) (hereinafter Yasui) in further view of Kim (US 20230191954 A1)(hereinafter Kim3).
Regarding claim 8, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim, Kim2, and Yasui specifically fails to disclose a system wherein the third processing unit is configured to analyze emotional information of an occupant based on seating information of the occupant included in the occupant detection information and set a pattern of the vibration signal based on the emotional information.
However, Kim3, which is in the same analogous art and that teaches about an apparatus and a method for controlling a vehicle seat, discloses a system wherein the third processing unit is configured to analyze emotional information of an occupant based on seating information of the occupant included in the occupant detection information(Kim3, paragraph 62, the processor 270 may be configured to recognize an emotional state of a user (e.g., a driver) using the emotional model ) and set a pattern of the vibration signal based on the emotional information(Kim discloses setting virtual sound bases on emotional state of the user. The sound is then used to determine the haptic/vibration pattern. Kim3, paragraph 65, when the vehicle state is the normal state and the seat sitting posture is the fatigue reduction posture, the processor 270 may be configured to control the haptic pattern and the haptic pressure of the seat back 10 and/or the neck pillow 120 based on the sound to provide a massage function. Kim3, paragraph 75, the processor 270 may be configured to select a sound-based haptic pattern via frequency and amplitude analysis of the sound. The processor 270 may be configured to select a haptic profile having a regular pattern and pressure or a haptic profile having a variable pattern and pressure. The haptic profile may correspond to a vibration excitation profile applied based on an emotional care resulting from an alternate vibration of the seat back and the neck pillow. The processor 270 may be configured to emotionally convert the music sound and/or the driving sound into a haptic pattern.).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Yasui with Kim3 to set the vibration pattern based on an emotional information of an occupant. By setting vibration pattern based on an occupant’s emotional information, it is possible to provide the user with healing emotion by relaxing their muscle tissues. ( Kim3, paragraph 93, the alternate vibrations of the seat back and the neck pillow may be controlled based on the sound in the vehicle driving condition, thereby maximizing the user's healing emotion. Kim3, paragraph 72, The haptic pressure may be selected based on biomechanical properties of a muscle and viscoelastic properties of a muscle tissue. The biomechanical properties of the muscle may include stiffness, and elasticity of the muscle. The muscle stiffness refers to resistance of the muscle tissue against an external force in an initial muscle state. That is, the stiffness refers to a magnitude of a force required to cause displacement of a muscle fiber tissue due to muscle contraction. The elasticity of the muscle refers to an ability to recover to an initial muscle shape after reduction or removal of the external force, and may be expressed as a vibration decrement).
Regarding claim 9, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management apparatus of claim 1(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the third processing unit is configured to analyze action information of the image information to set a pattern of the vibration signal(Kim3 discloses setting vibration pattern based on surrounding image/situation, which is similar to the image information. Kim3, paragraph 78, the processor 270 may be configured to determine a vibration pattern and a vibration excitation force based on the sound. The processor 270 may be configured to sense the sound reproduced in the vehicle. The processor 270 may be configured to use the detector 220 to detect vehicle environment information such as internal environment information of the vehicle and external environment information thereto. The vehicle environment information may include at least one of information such as a seat environment, a driving environment, a sound of a reproduced music, and/or a surrounding image (a surrounding situation)…The processor 270 may be configured to perform vibration compensation using the low-pass filtered and/or customized vibration-processed signal to determine the vibration pattern and the vibration excitation force. The processor 270 may be configured to implement seat vibration based on information such as the seat environment, the driving environment, the reproduced music sound, and/or the surrounding image).
Regarding claim 18, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C),
The combination of Kim, Kim2, and Yasui specifically fails to disclose a method wherein the generating of the sound signal includes:
analyzing, by the processor, emotional information of an occupant based on seating information of the occupant included in the occupant detection information; and
setting, by the processor, a pattern of the vibration signal based on emotional information.
However, Kim3, which is in the same analogous art and that teaches about an apparatus and a method for controlling a vehicle seat, discloses a method wherein the generating of the sound signal includes:
analyzing, by the processor, emotional information of an occupant based on seating information of the occupant included in the occupant detection information(Kim3, paragraph 62, the processor 270 may be configured to recognize an emotional state of a user (e.g., a driver) using the emotional model); and
setting, by the processor, a pattern of the vibration signal based on emotional information(Kim discloses setting virtual sound bases on emotional state of the user. The sound is then used to determine the haptic/vibration pattern. Kim3, paragraph 65, when the vehicle state is the normal state and the seat sitting posture is the fatigue reduction posture, the processor 270 may be configured to control the haptic pattern and the haptic pressure of the seat back 10 and/or the neck pillow 120 based on the sound to provide a massage function. Kim3, paragraph 75, the processor 270 may be configured to select a sound-based haptic pattern via frequency and amplitude analysis of the sound. The processor 270 may be configured to select a haptic profile having a regular pattern and pressure or a haptic profile having a variable pattern and pressure. The haptic profile may correspond to a vibration excitation profile applied based on an emotional care resulting from an alternate vibration of the seat back and the neck pillow. The processor 270 may be configured to emotionally convert the music sound and/or the driving sound into a haptic pattern).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kim and Yasui with Kim3 to set the vibration pattern based on an emotional information of an occupant. By setting vibration pattern based on an occupant’s emotional information, it is possible to provide the user with healing emotion by relaxing their muscle tissues. ( Kim3, paragraph 93, the alternate vibrations of the seat back and the neck pillow may be controlled based on the sound in the vehicle driving condition, thereby maximizing the user's healing emotion. Kim3, paragraph 72, The haptic pressure may be selected based on biomechanical properties of a muscle and viscoelastic properties of a muscle tissue. The biomechanical properties of the muscle may include stiffness, and elasticity of the muscle. The muscle stiffness refers to resistance of the muscle tissue against an external force in an initial muscle state. That is, the stiffness refers to a magnitude of a force required to cause displacement of a muscle fiber tissue due to muscle contraction. The elasticity of the muscle refers to an ability to recover to an initial muscle shape after reduction or removal of the external force, and may be expressed as a vibration decrement).
Regarding claim 19, the combination of Kim, Kim2 and Yasui teaches the virtual environment content management method of claim 11(Kim, paragraph 2, a virtual driving simulation device and a method for improving a sensation of immersion; Kim2, paragraph 12, the processing device may be configured to analyze an image obtained using a camera mounted on the outside of a vehicle to estimate a driving environment and may be configured to adjust volume of the virtual sound; Yasui, paragraph 66, the seating sensor 220C detects whether or not the passenger is seated in a seat provided with the seating sensor 220C on the basis of a pressure applied to the seating sensor 220C), wherein the generating of the sound signal includes analyzing, by the processor, action information of the image information to set a pattern of the vibration signal(Kim3 discloses setting vibration pattern based on surrounding image/situation, which is similar to the image information. Kim3, paragraph 78, the processor 270 may be configured to determine a vibration pattern and a vibration excitation force based on the sound. The processor 270 may be configured to sense the sound reproduced in the vehicle. The processor 270 may be configured to use the detector 220 to detect vehicle environment information such as internal environment information of the vehicle and external environment information thereto. The vehicle environment information may include at least one of information such as a seat environment, a driving environment, a sound of a reproduced music, and/or a surrounding image (a surrounding situation)…The processor 270 may be configured to perform vibration compensation using the low-pass filtered and/or customized vibration-processed signal to determine the vibration pattern and the vibration excitation force. The processor 270 may be configured to implement seat vibration based on information such as the seat environment, the driving environment, the reproduced music sound, and/or the surrounding image).
Prior Art of Record
The prior art made of record and not relied upon is considered pertinent to applicant’s
disclosure.
Hiromi(US 20230125250 A1) discloses an apparatus wherein the third processing unit is configured to analyze action information of the image information to set a pattern of the vibration signal(Hiromi discusses analyzing detected scene from an input image to determine vibration signal/pattern. Hiromi, paragraph 49, in the information processing method according to the embodiments, as described in the above-mentioned (1) to (5), a vibration pattern to be provided to a user is generated in accordance with an analysis processing result. Hiromi, paragraph 48, the information processing method according to the embodiment, (5) a specific scene is detected from an input image signal and an input sound signal. A frequency is divided by N by a pitch shift on the basis of a preliminarily-set vibration parameter in accordance with the detected scene ).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BESUFEKAD LEMMA TESSEMA whose telephone number is (571)272-6850. The examiner can normally be reached Monday - Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at 5712727298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BESUFEKAD LEMMA TESSEMA/Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights