DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements filed on 6/13/2023 and 9/20/2024 have been considered.
Drawings
The drawing s filed on 6/13/2023 are accepted.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-7, 11, 12, 15-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe (JP-2021/33944 A) in view of Wang (CN-114513622-A).
Regarding claim 1, Watanabe teaches a method for promoting communication between mobile platforms (Page 1 Abstract “To provide a communication device capable of visually and easily grasping from which vehicle around an own vehicle communication information is transmitted”), the method comprising:
receiving auto data that include speech from the operators of the mobile platforms; (Page 4 “Alternatively, the communication content may be voice data indicating an utterance of a driver or the like of a communication vehicle. In this case, the voice data can be collected and generated by a microphone mounted on the communication vehicle” and Page 9 “, in this case, there is a possibility that a plurality of voices overlap each other and the meaning of each voice cannot be understood. Further, in this case, even if the meaning of each voice can be understood, it may not be easy to understand which voice corresponds to the communication content from which communication vehicle”)
optionally, receiving transponder data indicating locations of each of the mobile platforms; (Page 4 “The communication information includes communication content information indicating the communication content and location information. The position information is information indicating the position of the communication vehicle that transmitted the communication information, specifically, the latitude and longitude. The position information can be acquired by a GPS receiver or the like mounted on the communication vehicle.”)
providing, by a processor (Fig. 13 [102]), a common display (Figs. 11 and 12) on display devices (Fig. 13 [6]) in each of the mobile platforms (Figs. 11 and 12), the common display including a terrain environment (Page 14 “The third embodiment is different from the other embodiments in that the communication vehicle symbol is superimposed and displayed on the map”) and mobile platform icons (Fig. 15 [90 & 92]) for each of the mobile platforms on the terrain environment indicating the locations of each of the mobile platforms; (Page 15 “The display control unit 350 displays the communication vehicle symbol 92Im on the map 80Im displayed on the screen 6a with the display size S1 determined according to the inter-vehicle distance L1. Further, the display control unit 350 displays the own vehicle symbol 90 Im on the screen 6a with the display size S1 determined according to the inter-vehicle distance L1”)
identifying, by the processor, a first of the mobile platforms associated with the first operator (Page 16 “Further, the communication device 100 according to the above-described embodiment identifies the communication vehicle by using the communication information including the position information of the communication vehicle” and ) and
indicating, on the common display by the processor, in real-time that the audio is associated with the first mobile platform. (Page 7 “Further, the display control unit 150 may display the communication content in the vicinity of the communication vehicle symbol on the screen of the display device 6. In this case, the display control unit 150 may display a balloon in the vicinity of the communication vehicle symbol on the screen and display the communication content in the balloon.”)
Watanabe discusses the use of voice recognition which requires databases to store information (Page 4 “Further, the text message may be generated by voice recognition from the utterance of the driver of the communication vehicle or the like.”), but differs from the claimed invention by not explicitly reciting voice profile data including voice profiles associated with the operators of each of the mobile platforms and analyzing, by the processor, the speech of the audio data to identify in real-time a first of the operators of the mobile platforms that is currently speaking by correlating the speech with one of the voice profiles of the voice profile data.
In an analogous art, Wang teaches a speaker detection method in a voice conferencing setting (Abstract) that includes storing voiceprint information for the speaker(s) (Page 7 “storing the voiceprint information the speaker”) and analyzing, by the processor, the speech of the audio data to identify in real-time a first of the operators of the mobile platforms that is currently speaking by correlating the speech with one of the voice profiles of the voice profile data (Page 14 “comparing the voiceprint information the speaker with the stored voiceprint information the first user, determining the confidence level of the first user of the speaker”) and displaying a visual cue. (Page 14 “the confidence degree determined by the voiceprint information the confidence degree determined by the lip information be weighted and processed, to comprehensively determine the speaker recognition (speaker is user A), then can display lip and small microphone under the user A icon, to remind other user speaker is user A.”)
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to be motivated to implement the invention of Watanabe after modifying it to incorporate the ability to utilize stored voiceprints to recognize the current speaker of Wang since utilizing voiceprints can help determine which speaker is speaking when there are multiple people connected in the conference. (Wang Page 6 “it needs to real-time determine the current speaker in the conference process, marking the current speaker in the video or switching the picture to the current speaker, the other people who are convenient to participate in the conference timely know who is speaking at present.”)
Regarding claim 5, Watanabe in view of Wang teaches visually differentiating, on the common display by the processor, the mobile platform icons of each of the mobile platforms to indicate a capability and/or type of mobile platform associated therewith. (Watanabe Page 5 “The vehicle type information may indicate the vehicle name (model) of the communication vehicle, or may indicate the model name (sedan, wagon, minivan, sports type, truck, etc.) of the communication vehicle. The vehicle ID is identification information for identifying the vehicle, and may be, for example, a vehicle registration number or a vehicle number (so-called license plate number). The symbol information is information indicating a communication vehicle symbol. The communication vehicle symbol indicated by the symbol information may be, for example, an icon having a shape corresponding to the vehicle type of the communication vehicle and a color icon corresponding to the color of the communication vehicle. Alternatively, the communication vehicle symbol indicated by the symbol information may be a thumbnail image of an image of the appearance of the communication vehicle.”)
Regarding claim 6, Watanabe in view of Wang teaches rendering, on the common display by the processor, visual representations of travel paths and/or travel patterns of each of the mobile platforms. (Pages 6-7 “For example, when the existing direction of the communication vehicle is the direction in front of the own vehicle, that is, the direction along the traveling direction, the display control unit 150 may display the communication vehicle symbol on the center of the screen. Further, when the existing direction of the communication vehicle is to the right front of the own vehicle, that is, to the right with respect to the traveling direction, the display control unit 150 may display the communication vehicle symbol on the upper right of the screen. Further, when the existing direction of the communication vehicle is behind the own vehicle, that is, in the direction opposite to the traveling direction, the display control unit 150 may display the communication vehicle symbol at the lower center of the screen.”)
Regarding claim 7, Watanabe in view of Wang teaches receiving, from one or more mobile platforms, safety critical data; (note: the broadest reasonable interpretation is just data, Watanabe Page 8 “the communication processing unit 160 may transmit information indicating the utterance emitted by the driver or the like and collected by the microphone 7 to the communication vehicle as the communication content”) and
displaying, on the common display by the processor, the safety critical data. (Watanabe Page 8 “the display control unit 150 may display a balloon or a symbol image showing the communication content of the communication vehicle corresponding to the communication vehicle symbol in the vicinity of the communication vehicle symbol.”)
Regarding claim 11, Watanabe in view of Wang teaches receiving visual data that includes one or more captured images (Page 4 “an image acquisition unit 122”, “The communication content is content for the driver of the communication vehicle to communicate with surrounding vehicles” and “Alternatively, the communication content may be a symbol image such as an icon, pictogram, or photographic image input or selected by the driver of the communication vehicle or the like”) one or more captured videos, or a streaming video feed in real-time from the first mobile platform; and
displaying, on the common display by the processor, in real-time the visual data. (Page 8 “Further, as described above, the display control unit 150 may display a balloon or a symbol image showing the communication content of the communication vehicle corresponding to the communication vehicle symbol in the vicinity of the communication vehicle symbol.”)
Regarding claim 12, the limitations of claim 12 are rejected as being the same reasons set forth above in claim 1. See additional structure:
Databases – Watanabe Fig. 3 [104] and Wang Fig. 14 [1402] and Page 16 “storing the sound information the speaker to the vocal print library”
Communication system – Watanabe Fig. 3 [106]
Display System/common display – Watanabe Fig. 3 [6]
Controller/processor – Watanabe Fig. 3 [102]
Regarding claims 15, 16, 17 and 20, the limitations of claims 15, 16, 17 and 20 are rejected as being the same reasons set forth above in claims 5-7 and 11.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Watanabe in view of Wang as applied to claim 1 above, and further in view of Ye et al. (US-2019/0279614 hereinafter, Ye).
Regarding claim 4, Watanabe in view of Wang teaches producing the voice profiles for the operators of each of the mobile platforms; (Wang Page 16 “determining the speaker from at least one user and storing the voiceprint information the speaker”)
training a speaker identification module to identify each of the operators using the voice profiles. (Wang Page 14 “comparing the voiceprint information the speaker with the stored voiceprint information the first user, determining the confidence level of the first user of the speaker” and “Optionally, if the voiceprint is information, determining that the speaker is not the first user, then the obtained voiceprint is information with the stored voice print information the other user, determining whether it is the other user who once said. if it is compared with the stored sound of other users to information the current speaking human second user”)
Watanabe in view of Wang teaches utilizing machine learning for lip movement identification (Wang Page 10 “Optionally, the face key point recognition the key point model and the lip key recognition model can be obtained by the neural network model or other network model training, no longer specifically to limit”), but differs from the claimed invention by not explicitly reciting testing the speaker identification module to ensure that the speaker identification module is capable of identifying each of the operators in real-time by correlating speech of the operators with the voice profiles, wherein the step of processing the speech of the audio data to identify in real-time which of the operators of the mobile platforms are currently speaking is performed with the speaker identification module.
In an analogous art, Ye teaches a method and system for advancements in acoustic to word modeling to improve speech recognition (Abstract) that includes training a speaker identification module to identify each of the operators using the voice profiles; (Page 6 [0044] “Training of an exemplary hybrid neural network model for speech recognition processing may comprise executing word-based modeling on samples of training data”)
testing the speaker identification module to ensure that the speaker identification module is capable of identifying each of the operators in real-time by correlating speech of the operators with the voice profiles, (Page 6 [0045] “training, during testing of the character-based modeling, may comprise detection of a character sequence for a portion of a speech signal that corresponds to an OOV token. Processing may further be applied to generate words from character sequences. Training is applied to obtain an exemplary speech recognition result by replacing an OOV token generated from the word-based modeling with a word generated from the character-based modeling that has the largest time overlap with the OOV token”)
wherein the step of processing the speech of the audio data to identify in real-time which of the operators of the mobile platforms are currently speaking is performed with the speaker identification module. (Page 7 [0048] “Such processing is useful in increasing processing efficiency during execution of decoding of speech signals to generate speech recognition results in real-time (or near real-time)”)
Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to be motivated to implement the invention of Watanabe in view of Wang after modifying it to incorporate the ability to utilize machine learning and training for speech recognition of Ye since real-time speech recognition reduces delay and increases user convenience.
Allowable Subject Matter
Claims 2, 3, 8-10, 13, 14, 18 and 19 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The Examiner was unable to find the combination of claims 1+2, 1+8, 12+13 or 12+18 in the prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW C SAMS whose telephone number is (571)272-8099. The examiner can normally be reached M-F 8:30-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Anderson can be reached at (571)272-4177. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Matthew C Sams/Primary Examiner, Art Unit 2646