DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 4, 5, 7-13, 16, 19 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments filed on 02/09/2026 with respect to claims 1-3, 14, 15, 17, 18 and 21 have been fully considered but they are not persuasive.
In re page 7, Applicant states that “Claim 16 was objected to. Applicant has amended claim 16 to recite "... cause the processor to..." instead of "... cause the processor to to..." Applicant respectfully requests for the withdrawal of the claim rejections”
(1) In response, the claim objection is herein withdrawn.
In re pages 7-8, Applicant states that “Applicant's invention is directed towards synchronizing the audio-visual data and medical data for the purpose of producing a composite video without relying on sound features like spoken terms which are distracting for medical personnel. Applicant's independent claim 1 recites "receiving the audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment; receiving the medical data recorded by a second device; classifying, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment; and synchronizing the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment.'' Wei, on the other hand, discusses "using] pre-trained models and re-training] the models with data from self-recorded videos to maximize the effectiveness of the deep learning algorithms to perform their bleeding detection and classification/estimation task... these models can be used in combination with an attention mechanism to visualize high-level adverse event features as a reasoning layer of bleeding and/or thermal injury detection and estimation," (see Wei, para [0116]). The Office Action alleges that the Wei's models can obtain audio data "from array microphones, which can be beamformed at a microphone level" (see Wei, para [0112]) to train the models on to "identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data)," or audio data, "and can simultaneously estimate the severity of the adverse event," (see Wei, para [0115]). Wei's method to detect bleeding and/or thermal injury by analyzing video (or audio) data using a deep learning models is not Applicant's claimed invention of "classifying, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment; and synchronizing the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment." Wei does not teach of synchronizing audio-visual data with medical data based on a "time of occurrence" of the sounds. Furthermore, Wei does not teach of the usage of sounds "classified as produced by the equipment." In fact, Wei teaches away from using sounds from medical equipment. Wei discusses extracting specific audio segments from her models, wherein the "feature extraction can include, for audio segments, generating labelling categories. These categories can be established based on various criterion, such as ease of labelling... and may further include approaches to exclude sounds that occur too often or are too common (e.g., oxygen saturation level device sounds)" (see Wei, para [0131]). Wei's models therefore exclude "sounds classified as produced by the equipment," as necessitated by Applicant's claims. For at least these reasons, Applicant respectfully submits that independent claims 1, 14, and 17 are patentable over Wei.”
(2) In response, the Examiner respectfully disagrees. For instance, WEI discloses the following” First, audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112. Second, video data can include camera data, which can be comprised of digital video frames where a number of digital images are displayed in rapid succession and provided at a particular rate (e.g., frames per second), the video data can be generated at various resolutions and filesizes as described in fig. 1 paragraph 112. Third, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event, the approach can include specific technical improvements which were identified during testing to aid with technical issues that arose during practical implementation as described in fig. 1 paragraph 115. Fourth, the encoder 22 can implement the bleeding detection and severity estimation described herein in some embodiments, the encoder 22 can provide video data and other data to another server for bleeding detection and severity estimation described herein in some embodiments as described in fig. 9 paragraph 299. Fifth, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container as described in fig. 9 paragraph 299. Sixth, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units as described in fig. 9 paragraph 299. Seventh, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299. Also, see fig. 9 paragraphs 110, 113, 288, 291-292, 301-302, 368. From the above passages, WEI indeed discloses the following claimed limitations of independent claim 1 that recites “receiving the audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment; receiving the medical data recorded by a second device.” See actual claim rejections below. As a result, the Applicant’s statements are unsupported by WEI.
Furthermore, WEI discloses the following” First, the system 100 can use pre-trained models and re-train the models with data from self-recorded videos to maximize the effectiveness of the deep learning algorithms to perform their bleeding detection and classification/estimation task as described in fig. 1 paragraph 116. Second, these models can be used in combination with an attention mechanism to visualize high-level adverse event features as a reasoning layer of bleeding and/or thermal injury detection and estimation as described in fig. 1 paragraph 116. Third, for supervised machine learning in relation to audio, a weak label approach as described in paragraph 358. Fourth, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358. Fifth, these length of these windows can be very small, typically only a few milliseconds long, as an example, an audio sampled at 44.1 Hz with a 512-frame window is only 12 ms long as described in paragraph 358. Also, see fig. 9 paragraphs 111, 130, 209, 288, 291, 292, 301, 302, 345-348, 368. From the above passages, WEI indeed discloses the following claimed limitations of independent claim 1 that recites “classifying, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment; and synchronizing the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment.” See actual claim rejections below. As a result, the Applicant’s statements are unsupported by WEI.
Furthermore, WEI discloses all the claimed limitations of independent claims 14 and 17 that recite similar claimed limitations.
In re page 8, Applicant states that “Applicant further respectfully submits that dependent claims 2-5, 7-13, 15, 18, and 20 are patentable by virtue of their respective direct and ultimate dependencies from allowable independent claims 1, 14, and 17, as well as by virtue of the patentably distinct subject matter that they recite.”
(3) In response, as discussed above in (2) with respect to independent claim 1 which is also applicable to the above Applicant’s arguments, WEI discloses all the claimed limitations of independent claims 1, 14 and 17.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 14 and 17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by WEI (US 2020/0265273 A1)(hereinafter WEI).
Re claim 1, WEI discloses a computer-implemented method for synchronizing audiovisual data and medical data in a medical procedure, the method comprising: receiving the audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment (see ¶s 110, 113 for receiving the audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368); receiving the medical data recorded by a second device (see ¶ 115 for receiving the medical data recorded by a second device (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 368); classifying, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (see ¶ 111 for classifying, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (i.e. the system 100 can use pre-trained models and re-train the models with data from self-recorded videos to maximize the effectiveness of the deep learning algorithms to perform their bleeding detection and classification/estimation task as described in fig. 1 paragraph 116, furthermore, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see paragraphs 130, 209, 345-348); and synchronizing the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (see ¶ 116 for synchronizing the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299, additionally, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368)
Re claim 2, WEI as discussed in claim 1 above discloses all the claim limitations with additional claimed feature wherein the equipment is a medical imaging system comprising the second device, and the medical data includes medical images (see ¶s 110, 112 for the equipment is a medical imaging system comprising the second device, and the medical data includes medical images (i.e. the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, furthermore, FIG. 4C illustrates an example of inputs and outputs of the system 450, images at the top, 451, 452, 453, 456, 457, and 458 show a few examples of different classes, the prediction plots 460, 465 indicate multi-class in one frame with multiple colors/shading in one vertical line, to achieve this, the vertical lines are plotted using different lengths, FIG. 4C shows in the top row instance frames from the video, each of which exemplifying one of five classes identified (background 451, blood 452, bleeding 453, background 456, thermal injury 457, burn 458), which is provided only as an example as described in paragraph 231). Also, see fig. 9 paragraph 299)
Re claim 14, WEI discloses a non-transitory computer readable medium, having stored instructions stored thereon that which, when executed by a processor, cause the processor to (i.e. the processor 104 can execute instructions in memory 108 as described in fig. 1 paragraph 142): receive audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing a medical procedure performed in a medical environment (see ¶s 110, 113 for receive audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368); receive medical data recorded by a second device (see ¶ 115 for receive medical data recorded by a second device (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 368); classify, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (see ¶ 111 for classify, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (i.e. the system 100 can use pre-trained models and re-train the models with data from self-recorded videos to maximize the effectiveness of the deep learning algorithms to perform their bleeding detection and classification/estimation task as described in fig. 1 paragraph 116, furthermore, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see paragraphs 130, 209, 345-348); and synchronize the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (see ¶ 116 for synchronize the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299, additionally, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368)
Re claim 17, WEI discloses a system for synchronizing audiovisual data and medical data in a medical procedure, the system comprising: a processor in communication with memory, the processor configured to (i.e. the processor 104 can execute instructions in memory 108 as described in fig. 1 paragraph 142): receive audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing a medical procedure performed in a medical environment (see ¶s 110, 113 for receive audio-visual data recorded by a first device, the audio-visual data including an audio channel and a video channel simultaneously capturing the medical procedure performed in a medical environment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368); receive medical data recorded by a second device (see ¶ 115 for receive medical data recorded by a second device (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299). Also, see fig. 9 paragraphs 288, 291-292, 368); classify, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (see ¶ 111 for classify, using one or more machine learning algorithms, one or more sounds from the audio channel of the audio-visual data produced by equipment in the medical environment (i.e. the system 100 can use pre-trained models and re-train the models with data from self-recorded videos to maximize the effectiveness of the deep learning algorithms to perform their bleeding detection and classification/estimation task as described in fig. 1 paragraph 116, furthermore, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see paragraphs 130, 209, 345-348); and synchronize the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (see ¶ 116 for synchronize the audio-visual data with the medical data based on a time of occurrence of the one or more sounds classified as produced by the equipment (i.e. audio data, for example, can be obtained from audio captured from array microphones (e.g., 8 channels), which can be beamformed at a microphone level as described in fig. 1 paragraph 112, furthermore, the system 100 implements a deep learning approach to identify or detect adverse events (such as bleeding and/or thermal injury) from a recording (e.g., video data), and can simultaneously estimate the severity of the adverse event as described in fig. 1 paragraph 115, moreover, the OR or Surgical encoder (e.g., encoder 22) may be a multi-channel encoding device that records, integrates, ingests and/or synchronizes independent streams of audio, video, and digital data (quantitative, semi-quantitative, and qualitative data feeds) into a single digital container, the digital data may be ingested into the encoder as streams of metadata and is sourced from an array of potential sensor types and third-party devices (open or proprietary) that are used in surgical, ICU, emergency or other clinical intervention units, these sensors and devices may be connected through middleware and/or hardware devices which may act to translate, format and/or synchronize live streams of data from respected sources as described in fig. 9 paragraph 299, additionally, for audio classification tasks, audio is often transformed into a spectrogram, which gets the frequency magnitudes for audio windows, which is a short slice of the audio as described in paragraph 358). Also, see fig. 9 paragraphs 288, 291-292, 301-302, 368)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 15, 18 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over WEI (US 2020/0265273 A1)(hereinafter WEI) as applied to claims 1, 2, 14 and 17 above, and further in view of Grimley (US 2015/0227694 A1)(hereinafter Grimley).
Re claim 3, WEI as discussed in claim 1 above discloses all the claimed limitations but fails to explicitly teach further comprising logging, in an event log, one or more events associated with the equipment, wherein the synchronizing comprises temporally matching the one or more sounds with the one or more events from the event log. However, the reference of Grimley explicitly teaches further comprising logging, in an event log, one or more events associated with the equipment, wherein the synchronizing comprises temporally matching the one or more sounds with the one or more events from the event log (see ¶s 41-42 for logging, in an event log, one or more events associated with the equipment, wherein the synchronizing comprises temporally matching the one or more sounds with the one or more events from the event log (i.e. the device 100 may prompt the user to obtain the code, thereby capturing equipment and/or data associated with the code into an event log 117, also, the device may include a wireless interface that is compatible with certain medical devices, for example a defibrillator, such that the device can obtain and record data captured by the medical device directly into the event log as described in figs. 3-4 paragraph 45, furthermore, the user activates the camera 104 and microphone 112 by either tapping on the start button 310 or by tapping any icon on the data entry screen 306, upon activation, the device begins to record video of the event that is being shown simultaneously behind the annotation icon graphics on the data entry screen 306, the software also obtains an audio record of the medical treatment event using the microphone 112, the device stores both video record and the audio record in memory 110 as described in fig. 7 paragraph 53, moreover, by capturing the identity of equipment used in the medical treatment event, any information that is being simultaneously captured by the equipment can also be captured or synchronized with the event log 117, then device 100 begins to wirelessly communicate with the identified medical device via the wireless transceiver 114, enabling device 100 to capture event data from the medical device directly, if equipped with a microphone, the defibrillator can also provide an audio record of the event to device 100, the data corresponding to the wireless signal transmissions is then recorded into the memory 110, all of the information acquired from another medical device may be synchronized by device 100 with the information recorded directly by device 100 and integrated in a time sequence in event log 117 as described in figs. 1, 17 paragraphs 71-72). Also, see figs. 5-6 paragraphs 46-48, 82)
Therefore, taking the combined teachings of WEI and Grimley as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (event log) into the system of WEI as taught by Grimley.
One will be motivated to incorporate the above feature into the system of WEI as taught by Grimley for the benefit of having a device and method which offers a simplified data entry interface for recording important information during a medical treatment event, wherein the interface should be capable of generating annotated event logs through the selection of contextually relevant icons on the touch screen, wherein the device preferably merges audio and video records of the event with the annotated event logs, wherein the device would be particularly useful in the documentation of CPR during cardiac arrest in order improve efficiency and have a user friendly interaction when generating annotated event logs through the selection of contextually relevant icons on the touch screen (see ¶ 10)
Re claim 15, the combination of WEI and Grimley as discussed in claim 3, and also, claim 14 above discloses all the claimed limitations of claim 15.
Re claim 18, the combination of WEI and Grimley as discussed in claim 3, and also, claim 17 above discloses all the claimed limitations of claim 18.
Re claim 21, the combination of WEI and Grimley as discussed in claim 3 above discloses all the claimed limitations but fails to explicitly teach wherein the events are logged while the medical images are captured by the second device. However, the reference of Grimley explicitly teaches wherein the events are logged while the medical images are captured by the second device (see ¶ 42 for the events are logged while the medical images are captured by the second device (i.e. the handheld computing device 100 comprises a touch screen display 102, a video camera 104 operable to capture a video record 2120, a memory 110 is operable to store an event log 117, a video record 118 of the event, and an audio record 119 of the event, preferably, the video record 118 and audio record 119 are correlated with or integrated into event log 117, such that event log 117 contains all relevant information about the event, the device of FIG. 3 is preferably arranged to allow a user to video a medical treatment event while simultaneously entering event data on the touch screen display as described in paragraph 41))
Therefore, taking the combined teachings of WEI and Grimley as a whole, it would have been obvious before the effective filing date of the claimed invention to incorporate this feature (event log) into the system of WEI as taught by Grimley.
Per claim 21, WEI and Grimley are combined for the same motivation as set forth in claim 3 above.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE M MESA whose telephone number is (571)270-1706. The examiner can normally be reached Monday-Friday 8:30AM-6:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
5/1/2026
/JOSE M. MESA/
Examiner
Art Unit 2484
/HUNG Q DANG/Primary Examiner, Art Unit 2484
Prosecution Insights