DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. DE10 2022 107 293.2, filed on 03/15/2023.
Status of Claims
Claims 1-15 and 24 are cancelled. Claims 16-23 and 25-31 filed on 10/29/2025 are presently examined. Claim 31 is new.
Response to Arguments
Regarding 35 U.S.C. 112(f), Applicant acknowledged 112(f) interpretation in remarks filed 10/29/2025.
Regarding 35 U.S.C. 101, Applicant’s amendments result in the withdrawal of the 35 U.S.C. 101 rejection.
Regarding 35 U.S.C. 102 and 103, Applicant's arguments filed 10/29/2025 have been fully considered but they are moot since the amendments result in a change in scope requiring a new ground of rejection. The 35 U.S.C. 102 rejection has been withdrawn and replaced with a 103 rejection. New reference Tin teaches tracking lip movement and adjusting music volume in response to the lip movement. Further, in regard to new claim 31, new reference Tatsuji teaches the consideration of the synchronization of the lip movement of the user with lyrics during karaoke.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“output module” in claims 16, 17 described in Applicant’s specification as [0008] “the output module may comprise the at least one loudspeaker.”, [0012] “output module may comprise the display apparatus”.
“storage module” in claim 18 described in Applicant’s specification as [0018] “a
storage module that stores, in particular permanently stores (as opposed to temporary buffer- storage in the case of streaming.”
“interface module” in claim 18 is described in Applicant’s specification as [0019] “The interface module can communicate with the external unit by means of a wireless connection (e.g., Bluetooth, WLAN, NFC, etc.).”
“artificial intelligence module configured to take” described in Applicant’s specification as [0081] “artificial intelligence module 130 comprises a trained algorithm”, [0065] “The system comprises one or more processors”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 16, 20, 23, and 26-30 are rejected under 35 U.S.C. 103 as being unpatentable over An et al. (US 20200073478 A1) in view of Tin et al (US 20200083856 A1), hereinafter referred to as An and Tin.
Regarding claims 16 and 29, An discloses an assistance system for a vehicle, comprising:
an output module designed for a music output based on musical acoustic data in a vehicle interior ([0059] “the vehicle … may include … speakers 250 configured to reproduce music”); and
an interior sensor system, including a camera and a microphone (microphone 222 and camera 36), configured to optically and/or acoustically detect at least one vehicle occupant and to provide corresponding detection data to the output module ([0079] “The sensing unit 210 may sense a user's emotional state during driving and acquire user emotion information indicating the user's emotional state.” [0059] “The controller 260 configured to control feedback devices provided in the vehicle 100 to control the user's emotional state based on the user emotion information received from the sensing unit 210”);
wherein the assistance system is configured to: take the detection data as a basis for determining a reaction of the at least one vehicle occupant in response to the music output ([0123] “sensing unit 210 may acquire the user emotion information indicating the user's emotional state using the sensors.” [0143] “when the determined user's current emotional state is anger, the controller 260 may control the speakers 250 to reproduce classical music.”); and
take the reaction of the at least one vehicle occupant in response to the music output as a basis for controlling at least one vehicle function in order to provide the at least one vehicle occupant with feedback in relation to the reaction in response to the music output ([0137] “When it is determined that the unit situation is not changed … acquire user emotion information indicating the user's emotional state again … the controller 260 may control the feedback devices again to reduce the negative emotion” [0145] “Referring to FIG. 6B, when it is determined that the emotional state of anger prior to control of the feedback devices is released based on the emotional state determined after control of the feedback devices, the display 270 may display an emotional state message 650 stating, ‘your emotional state of anger is released.’”);
An fails to explicitly disclose the reaction including a lip movement obtained by lip reading of the at least one vehicle occupant via the camera; control an output parameter of the music output based on the reaction of the at least one vehicle occupant in response to the music output, wherein the at least one output parameter includes a volume.
However, Tin teaches the reaction including a lip movement obtained by lip reading of the at least one vehicle occupant via the camera; control an output parameter of the music output based on the reaction of the at least one vehicle occupant in response to the music output, wherein the at least one output parameter includes a volume ([0008] “the camera can detect human speech by monitoring facial and/or lip movements of individuals. The automatic loudness control unit adjusts audio output from the audio output device. Further, the automatic loudness control unit can respond to human speech that is detected in the area.” [0009] “detecting human speech by monitoring at least one of vocal sounds, facial movements and lip movements of an individual and (d) applying automatic loudness control to adjust audio output for human speech.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Tin’s teaching of lip tracking the occupant of the vehicle and adjusting the volume of music in response to the lip movement. One would be motivated with reasonable expectation of success to adjust volume in response to lip movement in order to improve sound conditions for conversation (Tin [0115] “the system can identify human speech and improve sound conditions for conversation.”).
Regarding claim 20, An discloses The assistance system according to claim 16, wherein the reaction of the at least one vehicle occupant in response to the music output comprises at least one of the following elements:
a physical behavior of the at least one vehicle occupant, including a body expression, a body movement, a facial expression and/or a lip movement;
an acoustic performance of the at least one vehicle occupant with regard to the music output, including a volume, a rhythm and/or a register for singing along;
a synchronization between a singing along of the at least one vehicle occupant and the music output and/or the linguistic part of the music output; and
an emotional state of the at least one vehicle occupant ([0137] “When it is determined that the unit situation is not changed … acquire user emotion information indicating the user's emotional state again … the controller 260 may control the feedback devices again to reduce the negative emotion”).
Regarding claim 23, An discloses The assistance system according to claim 16, wherein the assistance system is configured to output the feedback to the at least one vehicle occupant audibly and/or visually and/or haptically ([0137] “display 270 may display an emotional state message 650 stating, ‘your emotional state of anger is released.’”).
Regarding claim 26, An discloses The assistance system according to claim 16, wherein the assistance system further comprises an intelligent personal assistant designed for an interaction with the at least one vehicle occupant, wherein the intelligent personal assistant is configured to determine the feedback in response to the music output ([at least FIG. 8A-B] the system takes in feedback according to the music output and interacts with the occupant. Intelligent personal assistant is broadly recited in this claim.).
Regarding claim 27, An discloses The assistance system according to claim 16, wherein the assistance system is configured to receive a music selection from the at least one vehicle occupant and/or to automatically make a music selection based on at least one circumstance parameter ([0008] “store situation information representing the numbers of expressions of respective emotional states in each unit situation” [0096] “a situation in which the vehicle 100 is driven during the morning rush hour, in the morning time, during the lunch hour, in the afternoon time, during the evening rush hour, at night, or in the middle of the night may become one unit situation.” Since the user emotional states are recorded according to situations, and the music output is selected based on user emotional data, the music selection is similarly based on the situation data, similar to circumstance parameters.).
Regarding claim 28, An discloses A motor vehicle comprising an assistance system according to claim 16 ([at least abstract] “a controller configured to control feedback devices provided in the vehicle”).
Regarding claim 30, An discloses A non-transitory storage medium storing instructions to be executed on one or more processors to perform an assistance method according to claim 29 ([0061] “The controller 260 may include at least one memory configured to store a program performing an operation, which is described below, and at least one processor configured to execute the stored program.”).
Claims 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over An in view of Tin as applied to claim 16 above, and further in view of Yasui et al. (US 20210150772 A1), hereinafter referred to as Yasui.
Regarding claim 18, An fails to explicitly disclose The assistance system according to claim 16, further comprising a storage module that has stored the musical acoustic data, wherein:
the storage module has further stored the lyrics data;
the storage module has permanently stored the musical acoustic data and/or the lyrics data;
and/or the storage module is integrated in the vehicle or an external unit connected to the vehicle, wherein the assistance system includes an interface module configured to receive the musical acoustic data and/or the lyrics data from the external unit.
However, Yasui teaches a storage module that has stored the musical acoustic data, wherein:
the storage module has further stored the lyrics data;
the storage module has permanently stored the musical acoustic data and/or the lyrics data;
and/or the storage module is integrated in the vehicle or an external unit connected to the vehicle, wherein the assistance system includes an interface module configured to receive the musical acoustic data and/or the lyrics data from the external unit ([0087] “The communicator 110 includes a network interface card (NIC), for example. The communicator 110 performs communication with the vehicle 200 via the network NW.” [0088] “content providing server 100 provides karaoke content to the passenger of the vehicle 200, karaoke content data 122 (for example, data of accompaniment music, lyrics, and the like) is stored in the storage 120.” External server storage is 120. [0063] “The storage 280 stores information received from the content providing server 100” Vehicle storage is 280.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Yasui’s teaching of an external content providing server that transmits the music and lyric data to the vehicle. One would be motivated with reasonable expectation of success to use an external content providing server in order to utilize the storage of an external server ([0086] “The content providing server 100 includes, for example, a communicator 110, a storage 120”).
Regarding claim 19, An fails to disclose The assistance system according to claim 16, wherein the assistance system is configured to receive the musical acoustic data and/or the lyrics data from an external unit and/or a streaming service.
However, Yasui teaches the assistance system is configured to receive the musical acoustic data and/or the lyrics data from an external unit and/or a streaming service ([0116] “The content providing server 100 transmits data such as an accompanying music, lyrics, and the like of the karaoke content to the vehicle 200.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Yasui’s teaching of an external content providing server that transmits the music and lyric data to the vehicle. One would be motivated with reasonable expectation of success to use an external content providing server in order to utilize the storage of an external server ([0086] “The content providing server 100 includes, for example, a communicator 110, a storage 120”).
Claims 21 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over An in view of Tin as applied to claim 16 above, and further in view of Sobhany (US 20200242421 A1), hereinafter referred to as Sobhany.
Regarding claim 21, An fails to disclose The assistance system according to claim 16, further comprising an artificial intelligence module configured to take the detection data as a basis for determining the reaction of the at least one vehicle occupant in response to the music output.
However, Sobhany teaches an artificial intelligence module configured to take the detection data as a basis for determining the reaction of the at least one vehicle occupant in response to the music output ([0067] “machine learning adaptation module 328 continuously learns about the user of the vehicle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Sobhany’s teaching of a machine learning adaptation model. One would be motivated with reasonable expectation of success to use a machine learning adaptation model in order to continuously improve the ability of the vehicle to respond or anticipate a user’s needs (Sobhany [0027] “utilize personalized models that learn a person's emotional states, habits, and reactions over time to improve the ability of the vehicle to respond to or anticipate a person's needs.”).
Regarding claim 22, An fails to disclose The assistance system according to claim 21, wherein the artificial intelligence module comprises a trained algorithm, wherein the trained algorithm is trained based on at least one of the following aspects:
video data from single and multiple persons singing to different songs;
musical acoustic data and/or lyrics data corresponding to the music output;
musical acoustic data and/or lyrics data similar to the music output; and
historical data with regard to earlier music outputs and reactions of vehicle occupants in response thereto.
However, Sobhany teaches the artificial intelligence module comprises a trained algorithm wherein the trained algorithm is trained based on at least one of the following aspects: musical acoustic data and/or lyrics data similar to the music output; musical acoustic data and/or lyrics data corresponding to the music output; historical data with regard to earlier music outputs and reactions of vehicle occupants in response thereto ([0090] “if … the driver is happy because she enjoys the music that is playing in the vehicle, the vehicle experience system 310 can select additional songs similar to the song that the driver enjoyed … the driver is currently frustrated due to heavy traffic but the vehicle experience system 310 has determined (based on historical data) that the driver will become happier if certain music is played”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Sobhany’s teaching of a machine learning adaptation model which uses historical data of past reactions to music, as well as the current and similar music that the occupant has enjoyed. One would be motivated with reasonable expectation of success to use historical data of past music and occupant reactions in order to improve a poor or maintain a good driver’s emotional state (Sobhany [0090] “the vehicle experience system 310 can select additional songs similar to the song that the driver enjoyed to ensure that the driver remains happy … the vehicle experience system 310 can play this music to change the driver's emotional state from frustration to happiness.”).
Claims 17 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over An in view of Tin as applied to claim 16 above, and further in view of Sofra (DE 102015014652 A1).
Regarding claim 17, An fails to disclose The assistance system according to claim 16, wherein the output module is further configured to visually output a linguistic part of the music output based on lyrics data.
However, Sofra teaches the output module is further configured to visually output a linguistic part of the music output based on lyrics data ([0019] “the user of the vehicle can listen to the radio and automatically have the lyrics of the piece of music displayed.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Sofra’s teaching of displaying lyrics along with the music. One would be motivated with reasonable expectation of success to display lyric of the outputted music in order to enable the occupants to sing along, providing karaoke functionality to the vehicle (Sofra [0034] “A driver 12 and/or another passenger 13 of the motor vehicle 1 now reads the text of the piece of music and sings the text 11 to the output melody of the piece of music. This provides a karaoke functionality of the motor vehicle 1”).
Regarding claim 25, An discloses The assistance system according to claim 16, wherein the at least one vehicle occupant is a single vehicle occupant ([0008] “a vehicle includes a sensing unit using at least one sensor and configured to acquire user emotion information”), or wherein the at least one vehicle occupant is two or more vehicle occupants.
An fails to disclose wherein a respective reaction in response to the music output is determined, and corresponding feedback is output, for each of the two or more vehicle occupants.
However, Sofra teaches a respective reaction in response to the music output is determined, and corresponding feedback is output, for each of the two or more vehicle occupants ([0038] “The noise atmosphere, in particular the voice, is then used, for example, to determine a current mood of the driver 12 and/or the other occupant 13” [0014] “The tonal agreement describes in particular how well a tone is hit by the occupants. Based on the degree of agreement, evaluation points can then be awarded” [0039] “Based on the sound atmosphere, in particular a degree of rhythmic and/or tonal agreement of the voice with the melody of the piece of music, or the degree of emotional mood, the current piece of music can then be recognized and stored, for example, as a favorite by the driver 12 and/or the other occupant 13.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Sofra’s teaching of using the mood and tonal agreement of the occupants in response to the music output. One would be motivated with reasonable expectation of success to detect occupants’ mood and tonal agreement with the music output in order to adapt the media content to suit the occupant and situation (Sofra [0040] “adapt the media content there to suit the occupant and the situation.”).
Claim 31 is rejected under 35 U.S.C. 103 as being unpatentable over An in view of Tin as applied to claim 16 above, and further in view of Tatsuji (JP 5574130 B2).
Regarding claim 31, An fails to disclose The assistance system according to claim 16, wherein the reaction further includes a synchronization between a singing along of the at least one vehicle occupant and the music output and the linguistic part of the music output.
However, Tatsuji teaches The assistance system according to claim 16, wherein the reaction further includes a synchronization between a singing along of the at least one vehicle occupant and the music output and the linguistic part of the music output ([0021] “comparing the recognized lip shape with the lip information corresponding to the singing reference information synchronized with the performance data”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify An with Tatsuji’s teaching of tracking lip shape in synchronization of lyrics. One would be motivated with reasonable expectation of success to track lip shape in synchronization with lyrics in order to determine whether the singer’s mouth shape is correct at the right point in time (Tatsuji [0021] “it is possible to determine whether the shape of the mouth of the singer is appropriate at the time of performance”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK R HEIM whose telephone number is (571)270-0120. The examiner can normally be reached M-F 9-6 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fadey Jabr can be reached at 571-272-1516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.R.H./Examiner, Art Unit 3668
/Fadey S. Jabr/Supervisory Patent Examiner, Art Unit 3668
Prosecution Insights