DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Remarks pages 9-10, filed 06/17/2026, with respect to the prior art rejections of amended claim(s) 1 and 15 under 35 U.S.C. 103, in regards to the amended limitation “wherein the classifier model and the generative model comprising the large language model are trained on a training data set that comprises, for each of a plurality of emergency scenes, corresponding captured sound data representing the emergency scene and a corresponding ground-truth descriptive summary of the emergency scene”, have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below) necessitated by Applicant’s amendment to the claim(s).
Applicant’s arguments, see Remarks page 10, filed 06/17/2026, with respect to the prior art rejections of amended claim(s) 1 and 15 under 35 U.S.C. 103, in regards to the amended limitation: “providing the descriptive summary of the emergency scene to the emergency responder, wherein providing the descriptive summary of the emergency scene to the emergency responder comprises a conversational assistant initiating an emergency communication with the emergency responder and providing the descriptive summary via the emergency communication”, have been fully considered but they are not persuasive.
On page 10 of Remarks, Applicant argues:
PNG
media_image1.png
342
752
media_image1.png
Greyscale
Examiner respectfully disagrees.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Wherein, Section 5.2 of Gong discloses “We observe LTU has superior performance in answering open-ended questions. In Table 5, we show LTU’s answers to questions asking about realistic audios. The audios are sampled from AudioSet evaluation set, which are extracted from YouTube videos and have not been used in LTU training.”, thus disclosing a model for audio processing, serving to process input audio and answer questions regarding the audio, such as regarding the audio classification and descriptions disclosed in Table 2 of Gong:
PNG
media_image2.png
313
935
media_image2.png
Greyscale
However, Gong fails to disclose the limitations “providing the descriptive summary of the emergency scene to the emergency responder, wherein providing the descriptive summary of the emergency scene to the emergency responder comprises a conversational assistant initiating an emergency communication with the emergency responder and providing the descriptive summary via the emergency communication”.
Section 1 Introduction of Wolf discloses: “Can we reduce the friction of form-filling for the citizen by auto-populating a report based on a photograph taken on their smartphone? This paper explores this question, and presents an approach that builds on publicly available data, models, and tools. As a proof-of-concept, we demonstrate how combining off-the-shelf tools and services including Google Teachable Machine, Microsoft Azure Cognitive Image API, and OpenAI’s GPT models can accelerate the process of analysing incoming incident data, extract valuable information from submitted images, and help generate an automated hazard impact report. With this approach, we aim to reduce friction for citizens in time critical situations when reporting incidents and support emergency services in their response through (near-) real-time image analysis showing the impact of the hazard.”, wherein the system proposed by Wolf serves to generate a climate hazard report, through the analysis and extraction of data from the captured images, and the providing of the generated report to emergency services.
Further, 2.2 Design and development of an automated climate hazard report through computer vision and AI-based text generation of Wolf discloses: “OpenAI’s GPT models (accessed through Lex.page) to generate structured text descriptions based on keywords and predicted class labels provided by the Azure Cognitive Image API. This involves providing the model’s content and posing a series of specific questions…OpenAI GPT-3, and later GPT-4, were chosen for their comprehension abilities and ease of prompting. We address architectural considerations in deployments of such models in Section 3.3…The next step, data transformation, involves reverse geocoding the incident coordinates, classifying the event type, and recognizing objects. This transformed data is then fed through generative natural langugage models to create the final report.”, wherein the processing of the transformed data, extracted from the captured images, through the prompting of the OpenAI GPT generative model, constitutes a conversational model providing emergency communication with the hazard impact report. In addition, the system itself, once the user captures the input images, performs the actions of report generation and sending it to emergency services as is disclosed by section 2.1 Reducing friction using camera-first reporting of Wolf: “we propose using smartphone cameras as a primary method for hazard reporting, where the impacted citizen takes a photograph, and the remainder of the process is offloaded to our system.”.
Thus, Wolf in view of Gong discloses the limitations “providing the descriptive summary of the emergency scene to the emergency responder, wherein providing the descriptive summary of the emergency scene to the emergency responder comprises a conversational assistant initiating an emergency communication with the emergency responder and providing the descriptive summary via the emergency communication”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-5, 8-9, 11-15, 17-19, 22-23, and 25-28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wolf et al. (Camera-First Form Filling: Reducing the Friction in Climate Hazard Reporting) hereinafter referenced as Wolf, in view of Gong et al. (Listen, Think, and Understand) hereinafter referenced as Gong, and Kim et al. (AudioCaps: Generating Captions for Audios in The Wild) hereinafter referenced as Kim.
Regarding claim 1, Wolf discloses: A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations (Wolf: Section 4: “we demonstrate how such data pipelines can be built using off-the-shelf AI services, including Google’s Teachable Machine, Microsoft Azure Cognitive Image API, and OpenAI’s GPT models.”) comprising: receiving data representing an emergency scene, the data comprising image data captured by a camera in communication with a user device present at the emergency scene (Wolf: Figure 1: Input data; Section 2.2: “Figure 1 shows the flow of the individual steps from a climate hazard (Hurricane Ian used here) occurs by a citizen who then captures the hazard in a photo which includes meta-information such as an image description, date, time, and the location of the incident.”);
processing, using a classifier model, the received data to identify aspects at the emergency scene; detecting an emergency at the emergency scene based on the identified aspects at the emergency scene (Wolf: Figure 1: Data transformation; Section 2.2: “An image classifier using MobileNet via Google’s no-code Teachable Machine tool and a collection of images of different climate hazards. This allows us to classify the incoming images into different categories of natural hazards, such as large-scale or small-scale hazards, and different types, including floods and earthquakes. An off-the-shelf Microsoft Azure Cognitive Image API to detect and extract objects from images indicating if urban assets, environmental infrastructure, and people might be located in spatial proximity and could be impacted by the hazard…The next step, data transformation, involves reverse geocoding the incident coordinates, classifying the event type, and recognizing objects.”);
and in response to detecting the emergency at the emergency scene: generating, using a generative model configured to receive the image data representing the emergency scene, a descriptive summary of the emergency scene (Wolf: Figure 1: Output data; Tables 2 & 3; Section 2.2: “The next step, data transformation, involves reverse geocoding the incident coordinates, classifying the event type, and recognizing objects. This transformed data is then fed through generative natural langugage models to create the final report.”), the generated descriptive summary of the emergency scene providing detailed information about the emergency scene to assist an emergency responder in preparing for the emergency scene upon arrival (Wolf: 1 INTRODUCTION: “we demonstrate how combining off-the-shelf tools and services including Google Teachable Machine, Microsoft Azure Cognitive Image API, and OpenAI’s GPT models can accelerate the process of analysing incoming incident data, extract valuable information from submitted images, and help generate an automated hazard impact report. With this approach, we aim to reduce friction for citizens in time critical situations when reporting incidents and support emergency services in their response through (near-) real-time image analysis showing the impact of the hazard.”); and
providing the descriptive summary of the emergency scene to the emergency responder, wherein providing the descriptive summary of the emergency scene to the emergency responder comprises a conversational assistant initiating an emergency communication with the emergency responder and providing the descriptive summary via the emergency communication, (Wolf: Figure 1:
PNG
media_image3.png
356
848
media_image3.png
Greyscale
;
Section 1: “we demonstrate how combining off-the-shelf tools and services including Google Teachable Machine, Microsoft Azure Cognitive Image API, and OpenAI’s GPT models can accelerate the process of analysing incoming incident data, extract valuable information from submitted images, and help generate an automated hazard impact report. With this approach, we aim to reduce friction for citizens in time critical situations when reporting incidents and support emergency services in their response through (near-) real-time image analysis showing the impact of the hazard. Overall, this study contributes to facilitating multi-agency reporting of climate hazard impacts.”
Section 3.1: “While existing M/ETHANE reports provide vital information to the responding agencies, we show how we can further enhance their impact assessment through the analysis of imagery data. AI-assisted image analysis can provide a quicker assessment of the scope of response required, which would have taken more time and effort if responders had extracted and analysed image data manually.”; Wherein the automated hazard impact report generated by the GPT model constitutes a conversational assistant initiating an emergency communication with the emergency responder),
wherein the generative model comprises a large language model (Wolf: Figure 1: Output data; Tables 2 & 3; Section 2.2: “Our proof-of-concept workflow is as follows…OpenAI’s GPT models (accessed through Lex.page) to generate structured text descriptions based on keywords and predicted class labels provided by the Azure Cognitive Image API. This involves providing the model’s content and posing a series of specific questions.”; Wherein transformed image data are processed by the OpenAI GPT generative model, which is a large language model).
Wolf does not disclose expressly: the data comprising sound data captured by a microphone in communication with a user device present at the emergency scene; and in response to detecting the emergency at the emergency scene: generating, using a generative model configured to receive the sound data representing the emergency scene, a descriptive summary of the emergency scene, and wherein the classifier model and the generative model comprising the large language model are trained on a training data set that comprises, for each of a plurality of emergency scenes, corresponding captured sound data representing the emergency scene and a corresponding ground-truth descriptive summary of the emergency scene.
Thus, Wolf does not disclose expressly: a classifier model and a generative model comprising a large language model, wherein the classifier model and the generative model comprising the large language model are configured to process captured sound data and are trained on a training data set that comprises sound data and a corresponding ground-truth descriptive summary for a plurality of emergency scenes.
Gong discloses: a model titled Listen, Think, and Understand (LTU), wherein the model processes audio data and performs audio perception and reasoning tasks (Gong: Abstract: “Therefore, we ask the question: can we build an AI model that has both audio perception and a reasoning ability? In this paper, we propose a novel audio foundation model, called LTU (Listen, Think, and Understand)…it exhibits remarkable reasoning and comprehension abilities in the audio domain.”). Wherein the LTU model is a generative AI system (Gong: 3 The OpenAQA Dataset: “Since LTU is a generative AI system and is anticipated to possess a wide range of capabilities for diverse audio tasks, existing datasets are not sufficient to train LTU.”) comprising a large language model (Gong: 2 LTU Model Architecture: “LLaMA Large Language Model. In this paper, we use the LLaMA-7B large language model [5] with Vicuna [9] instruction following training. LLaMA is pretrained on a combination of natural and programming language corpora in a self-supervised manner.”), wherein the LTU model processes the received data in order to perform audio classification (Gong: 3.1 Close-Ended Audio Question-Answer Generation: “We include four closed-ended audio tasks…Classification: The question for this task is “classify the sound events in the audio clip” and its GPT-assisted paraphrases; The answer is a list of sound class names that the audio clip contains.”) and descriptive summary generation (Gong: 5.1 Closed-Ended Audio Task Experiments: “Summarization: As an open-ended AQA model, LTU also exhibits a strong generalization ability across closed-ended datasets and tasks. Notably, LTU directly outputs its prediction in text, automatically filters out non-prominent sounds in the prediction, and requires no pre-defined label set for inference. These all make LTU a more practical system for real-world applications.”) tasks. The LTU model is trained on a training data set that comprises, for each of a plurality of scenes, corresponding captured sound data representing the scene and a corresponding ground-truth descriptive summary of the scene (Gong: 1 Introduction: “In order to train LTU, we create a new dataset called OpenAQA-5M that combines 8 mainstream audio datasets and formats all data as (audio, question, answer) tuples.”;
Table 2: “Question: What mood or atmosphere does the audio clip convey? Answer: The audio clip conveys a sense of urgency, due to the sound of the ambulance siren, mixed with a sense of a busy, noisy environment, due to the sound of traffic noise.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the audio processing LTU model taught by Gong into the emergency report generation system disclosed by Wolf, in order to incorporate audio captured by the user’s smartphone into the emergency report generation. The suggestion/motivation for doing so would have been “LTU understands the scene and can connect sounds to actions. In Table 5…Sample 3, LTU understands it is a dangerous scene and suggests seeking shelter when a gunshot is heard” (Gong: 5.2 Open-ended Audio Tasks Experiment). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Wolf in view of Gong does not disclose expressly: wherein the classifier model and the generative model comprising the large language model are trained on a training data set that comprises, for each of a plurality of emergency scenes, corresponding captured sound data representing the emergency scene and a corresponding ground-truth descriptive summary of the emergency scene.
Kim discloses: a training data set that comprises, for each of a plurality of emergency scenes, corresponding captured sound data representing the emergency scene and a corresponding ground-truth descriptive summary of the emergency scene (Kim: Figure 6:
PNG
media_image4.png
482
1121
media_image4.png
Greyscale
;
Section: 1 Introduction: “For example, suppose that suddenly sirens are ringing in the downtown area. As a natural reaction, people may notice the presence of an emergency vehicle, even though they are unable to see any flashing lights nor feel the rush of wind from a passing vehicle. Instead of simply tagging this sound as ambulance or siren, it is more informative to describe which direction the sound is coming from or whether the source of the sound is moving closer or further away, as shown in Figure 1. To that end, we address the audio captioning problem for audios in the wild, which has not been studied yet, to the best of our knowledge…The contributions of this work are as follows. 1. To the best of our knowledge, this work is the first attempt to address the audio captioning task for sound in the wild. We contribute its first large-scale dataset named AudioCaps, which consists of 46K pairs of audio clips and text description.”; Wherein the data entries within the AudioCaps dataset containing fire, explosions, and ambulance/fire engine siren data, all constitute emergency scenes.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the AudioCaps dataset taught by Kim for the training of the audio processing LTU model disclosed by Wolf in view of Gong. The suggestion/motivation for doing so would have been “In contrast to audio classification, which uniquely map the audio to a set of labels, our task generates a descriptive sentence. Hence, it needs to not only detect salient sounds of classes but also explores their states, properties, actions or interactions.” (Kim: Section: 2 Related Work; Wherein the text descriptions includes further details regarding the states, properties, actions and interactions of the captured sounds). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Wolf in view of Gong with Kim to obtain the invention as specified in claim 1.
Regarding claim 3, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein providing the descriptive summary of the emergency scene to the emergency responder comprises providing, using a conversational assistant, the descriptive summary of the emergency scene to the emergency responder (Gong: 5.2 Open-ended Audio Tasks Experiment: “We observe LTU has superior performance in answering open-ended questions. In Table 5, we show LTU’s answers to questions asking about realistic audios. The audios are sampled from AudioSet evaluation set, which are extracted from YouTube videos and have not been used in LTU training.”)
(Wolf: Figure 1: Output data; Tables 2 & 3; Section 2.2: “OpenAI’s GPT models (accessed through Lex.page) to generate structured text descriptions based on keywords and predicted class labels provided by the Azure Cognitive Image API. This involves providing the model’s content and posing a series of specific questions” Section 3.2: “the suggested M/ETHANE report template can be enhanced and customized by providing additional questions relevant to other emergency agencies”; Wherein the GPT model disclosed by Wolf and the LTU model disclosed by Gong creating a report based on a template of questions asked regarding their respective data constitutes using a conversational assistant.).
Regarding claim 4, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the generative model is further configured to receive the aspects of the emergency scene when generating the descriptive summary of the emergency scene (Wolf: Figure 1: Input data; Section 1: “With this approach, we aim to reduce friction for citizens in time critical situations when reporting incidents and support emergency services in their response through (near-) real-time image analysis showing the impact of the hazard”; Wherein data submitted by a user, including images and audio, for the purposes of incident reporting is processed when generating the descriptive summary).
Regarding claim 5, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the operations further comprise: receiving, from a person, an indication of the emergency, wherein detecting the emergency at the emergency scene is further based on receiving the indication of the emergency (Wolf: Figure 1: Input data; Section 1: “With this approach, we aim to reduce friction for citizens in time critical situations when reporting incidents and support emergency services in their response through (near-) real-time image analysis showing the impact of the hazard”; Wherein a user submits captured data, including images and audio, for the purposes of reporting an incident, wherein the data is analyzed in order to extract information regarding the emergency).
Regarding claim 8, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the descriptive summary is generated to assist the emergency responder in preparing for the emergency scene upon arrival (Wolf: Figure 1: Action; Abstract: “To quickly and correctly assess the situation and deploy resources, emergency services often rely on citizen reports that must be timely, comprehensive, and accurate…By building on existing computer vision and natural language models, we demonstrate the automated generation of a full-form hazard impact assessment report from a single photograph.”; Section 4: “Building shared situational awareness is critical for emergency response agencies to respond effectively to an incident…This paper aims to simplify and accelerate this process to gather higher-quality information and reduce the burden on emergency services.”).
Regarding claim 9, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the descriptive summary comprises at least one of: states of one or more vehicles involved in an accident at the emergency scene; states of one or more airbags; locations of one or more vehicles at the emergency scene; a health status of one or more persons involved in the accident at the emergency scene; locations of one or more persons involved in the accident; a presence of fire; a presence of water; a presence of a roadway; damage to a roadway; a terrain topology; a presence of a leaking fluid; debris on a roadway; a roadblock condition; sounds at the emergency scene comprising speaking, crying, moaning, or barking; a description of surroundings; or a presence of weapons (Wolf: Tables 2 & 3; Wherein the reports generated based on the transformed input image’s data includes descriptions regarding vehicle states and locations, people present and their conditions, flood, road locations, roadblocks, and surroundings.)
(Gong: Table 5: “Real samples of LTU open-ended question answering. Answers are directly taken from the model output without modification. Notes in grey square brackets are not input to the model…Sample 3. Gun Shot Sound Question: Describe the audio clip with a sentence. [Closed-Ended Question] Answer: Gunshots and footsteps are heard, followed by a man speaking and a beep.”; Wherein audio and image data submitted by the user is processed when generating the descriptive summary)
(Kim: Figure 6:
PNG
media_image4.png
482
1121
media_image4.png
Greyscale
;
Section: 1 Introduction: “For example, suppose that suddenly sirens are ringing in the downtown area. As a natural reaction, people may notice the presence of an emergency vehicle, even though they are unable to see any flashing lights nor feel the rush of wind from a passing vehicle. Instead of simply tagging this sound as ambulance or siren, it is more informative to describe which direction the sound is coming from or whether the source of the sound is moving closer or further away, as shown in Figure 1.”).
Regarding claim 11, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the received capturing the data representing the emergency scene further comprises image data captured by a camera in communication with the user device present at the emergency scene (Wolf: Figure 1: Input data; Section 2.2: “A demonstration workflow for generating a pre-filled report using only a smartphone camera photograph is described herein and visualized in Figure 1. We assume that photographs are geotagged – either the smartphone embeds this information in the EXIF (the standard for metadata in media containers) fields or attaches the coordinates when uploading the photograph.”; Wherein the usage of geotagged photograph locations for report generation discloses a user present at the scene.).
Regarding claim 12, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 11, wherein providing the descriptive summary of the emergency scene to the emergency responder further comprises providing the image data captured by the camera to the emergency responder (Wolf: Section 3.3: “Third, we believe that reports containing images are more useful. If either cloud models or image-including reports are used, then both can rely on the same uploaded image.”).
Regarding claim 13, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 11, wherein the image data comprises an image of a person associated with the user device to facilitate identification of the person by the emergency responder (Wolf: Tables 2 & 3: “Can you identify people?”; Section 3.3: “Third, we believe that reports containing images are more useful. If either cloud models or image-including reports are used, then both can rely on the same uploaded image.”; Wherein the input images are used to identify people, including the user, as is disclosed in the report generating template questions shown in Tables 2 & 3, and wherein the final report includes the input images.).
Regarding claim 14, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein the received data representing the emergency scene further comprises sensor data from one or more sensors in communication with the data processing hardware, the one or more sensors comprising at least one of a speed sensor, an altitude sensor, an accelerometer, a braking sensor, a position sensor, a temperature sensor, or a light sensor (Wolf: Table 1; Section 2.2: “We assume that photographs are geotagged – either the smartphone embeds this information in the EXIF (the standard for metadata in media containers) fields or attaches the coordinates when uploading the photograph.”; Wherein the smartphone geotagging photographs constitutes the usage of a position sensor.).
As per claim(s) 15, arguments made in rejecting claim(s) 1 are analogous. In addition, Abstract of Wolf discloses the usage of computer vision and natural language model, implying the usage of a system comprising data processing hardware and memory hardware, storing instructions, in communication with the data processing hardware.
As per claim(s) 17, arguments made in rejecting claim(s) 3 are analogous.
As per claim(s) 18, arguments made in rejecting claim(s) 4 are analogous.
As per claim(s) 19, arguments made in rejecting claim(s) 5 are analogous.
As per claim(s) 22, arguments made in rejecting claim(s) 8 are analogous.
As per claim(s) 23, arguments made in rejecting claim(s) 9 are analogous.
As per claim(s) 25, arguments made in rejecting claim(s) 11 are analogous.
As per claim(s) 26, arguments made in rejecting claim(s) 12 are analogous.
As per claim(s) 27, arguments made in rejecting claim(s) 13 are analogous.
As per claim(s) 28, arguments made in rejecting claim(s) 14 are analogous.
Claim(s) 6-7 and 20-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wolf in view of Gong and Kim, and further in view of Gross (US10699580B1).
Regarding claim 6, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 4.
Wolf in view of Gong and Kim does not disclose expressly: wherein the operations further comprise: receiving, from a vehicle involved in the emergency, an indication of the emergency, wherein detecting the emergency at the emergency scene is further based on receiving the indication of the emergency.
Gross discloses: a vehicle detecting an emergency, and generating an incident report containing collected sensor data to send to an emergency responder (Gross: Col 12: Lines: 23-28: “Activities that can activate the accident reporting system include, but are not limited to, other vehicles hitting the vehicle 103, humans and/or animals touching and/or damaging the vehicle 103, vandalism to the vehicle 103, theft of the vehicle 103, and/or foreign objects falling on the vehicle 103.”;
Col 12: Lines: 58-61: “When an incident is detected, the vision system of the vehicle 103 takes pictures and/or video recordings of surroundings and the audio system records sounds made around the time of detection or interference. The data collected from detection of the incident can be recorded analyzed and used to generate an incident report. This report is sent to the user via the user device 106. The incident report can contain screen shots and/or video of the incident with probable perpetrators along with any audio that was recorded using an installed microphone during the incident. The incident report also can be sent to an emergency responder user device 106 a.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the vehicle with automatic incident detection and incident reporting system taught by Gross as an additional input device for the hazard reporting system disclosed by Wolf in view of Gong and Kim. The suggestion/motivation for doing so would have been “But if the owner is not available, not available to respond quickly, or is incapacitated, it will be necessary to be able to transfer or hand off control of the automated vehicle to an emergency responder without receiving approval from the owner. Waiting to receive approval from the owner could result in injured persons not receiving timely attention and care. It also could create risk of further accidents and injuries, where a vehicle cannot be promptly moved out of the way of traffic.” (Gross: Col 2-3: Lines: 62-03: Wherein if a user is not able to contact emergency services for help, the vehicle may do so in order to the people involved to receive help as soon as possible.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Wolf in view of Gong and Kim with Gross to obtain the invention as specified in claim 6.
Regarding claim 7, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1.
Wolf in view of Gong and Kim does not disclose expressly: wherein providing the descriptive summary of the emergency scene to the emergency responder further comprises providing the sound data captured by the microphone to the emergency responder.
Gross discloses: the capturing of audio data during a detected incident for the generation of an incident report, wherein the generated incident report, containing captured audio and image data, is provided to an emergency responder (Gross: Col 12: Lines: 52-61: “When an incident is detected, the vision system of the vehicle 103 takes pictures and/or video recordings of surroundings and the audio system records sounds made around the time of detection or interference. The data collected from detection of the incident can be recorded analyzed and used to generate an incident report. This report is sent to the user via the user device 106. The incident report can contain screen shots and/or video of the incident with probable perpetrators along with any audio that was recorded using an installed microphone during the incident. The incident report also can be sent to an emergency responder user device 106 a.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the known technique of providing an emergency responder collected audio and image data disclosed by Gross into the emergency report generation system disclosed by Wolf in view of Gong and Kim by providing the emergency responder the generated descriptive summary along with collected image and audio data. The suggestion/motivation for doing so would have been “The incident report can contain screen shots and/or video of the incident with probable perpetrators along with any audio that was recorded using an installed microphone during the incident.” (Gross: Col 12: Lines: 58-61; Wherein the collected data may contain information that may be better expressed as audio/images rather than text.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Wolf in view of Gong and Kim with Gross to obtain the invention as specified in claim 7.
As per claim(s) 20, arguments made in rejecting claim(s) 6 are analogous.
As per claim(s) 21, arguments made in rejecting claim(s) 7 are analogous.
Claim(s) 10 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wolf in view of Gong and Kim, and further in view of Do et al. (Virtual assistant for first responders using natural language understanding and optical character recognition) hereinafter referenced as Do.
Regarding claim 10, Wolf in view of Gong and Kim discloses: The computer-implemented method of claim 1, wherein: the descriptive summary comprises text (Wolf: Figure 1: Output data; Section 2.2: “OpenAI’s GPT models (accessed through Lex.page) to generate structured text descriptions based on keywords and predicted class labels provided by the Azure Cognitive Image API. This involves providing the model’s content and posing a series of specific questions”)
(Gong: 5.1 Closed-Ended Audio Task Experiments: “Audio Classification: Since LTU directly outputs audio label names or descriptions in text form instead of label index, in order to benchmark and compare it with existing models, we encode the LTU output and the evaluation dataset label names using a text encoder (gpt-text-embedding-ada), and compute the cosine similarity between the text embedding of LTU output and each label…We tested two prompts “classify the sound events in the audio clip” and ’write an audio caption describing the sound’, which let the model output a list of sound classes and a sentence describing the sound, respectively (examples shown in Table 5).”).
Wolf in view of Gong and Kim does not disclose expressly: the operations further comprise providing the text to a text-to-speech (TTS) system, the TTS system configured to convert the text into TTS audio data that conveys the descriptive summary as synthetic speech; and providing the descriptive summary of the emergency scene to the emergency responder comprises providing the TTS audio data to the emergency responder.
Do discloses: a virtual assistant for first responders, allowing for first responders to ask questions to, and receive information from, a chatbot regarding certain scenarios. Wherein the chatbot is able to respond to the user using text and text-to-speech (Do: Abstract: “We have developed a user-friendly application through understanding the hazardous material database, first aid safety guidelines and observing the process of first responders who access this information in the field. We created the Trusted and Explainable Artificial Intelligence for Saving Lives (TruePAL) virtual assistant using Dialogflow1 and TensorFlow2 paired with EasyOCR.3 The chatbot supports first responders by providing voice interaction which helps limit additional steps such as browsing through multiple categories when searching for information.”; 4.3.3 Chatbot: “The Chatbot allows first responders to talk using colloquial and domain specific language using either text or voice to provide information and instructions for dealing with various scenarios. Responses by the chatbot are given both as text and text to speech.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to process the final report disclosed by Wolf in view of Gong and Kim with the text-to-speech system disclosed by Do. The suggestion/motivation for doing so would have been “…to generate a visual and text-to-speech response. The functionality of the TruePAL virtual assistant allows for flexible use in both noisy and less noisy environments. This accommodation creates a better and trusted user experience.” (Do: 5.1 Conversation Design). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Wolf in view of Gong and Kim with Do to obtain the invention as specified in claim 10.
As per claim(s) 24, arguments made in rejecting claim(s) 10 are analogous.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights