Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 3 is objected to because of the following informalities:
Claim 3 ends in a comma rather than a period.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claims 1-13 are drawn to an apparatus, claim 14 is drawn to a method, and claim 15 is drawn to a non-transitory computer readable medium, each of which is within the four statutory categories.
Step 2A(1)
Claim 1 recites, in part, performing the steps of:
for each of a plurality of data modalities, process data of the respective data modality to generate one or more labels, each identifying an event occurring at a time on the common timeline and indicated by the processed data,
the data having been collected during a clinical procedure and belonging to a plurality of data modalities, at least one of the data modalities being an imaging data type and a further of the data modalities being an additional data type other than imaging data, wherein the data is provided with reference to a common timeline over which the data is collected, and
processing the labels for each of the identified events based on the relative time of occurrence of the events to obtain an output indicative of a further medical event.
These steps constitute a series of concepts performable in the human mind, and therefore fall within the scope of an abstract idea in the form of a mental process. Fundamentally the process is that of identifying and labeling the occurrence of events from a plurality of data modalities collected during a clinical procedure, and using the relative time that the labeled events occurred to determine a further medical event. These steps constitute a series of observing the data, evaluating and rendering judgement based on the events and their timeline within the data, and forming an opinion about a further medical event indicated by the timeline of events, and could be performed within the human mind by a clinician observing the data.
Independent claims 14 and 15 recite similar limitations and also recite an abstract idea under the same analysis.
Step 2A(2)
This judicial exception is not integrated into a practical application because the additional elements within the claims only amount to:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
Claim 1 further recites the additional element of processing circuitry recited as configured to perform the subsequent data analysis steps.
Claim 15 recites the additional events of a) a non-transitory computer readable medium recited as comprising computer-readable instructions, and b) a processor recited as executing the instructions to perform the subsequent data analysis steps.
Page 4 line 23 – page 5 line 32 describe an apparatus for performing the disclosed functions, where the apparatus includes a user mobile device, a personal computer, a server, or “some other form of device.” The apparatus is disclosed as having a data processor as well as non-transitory memory devices storing computer readable instructions for execution by the data processor. The processing circuitry, non-transitory computer readable medium, and processor are each therefore construed as encompassing generic computing components.
The processing circuitry, non-transitory computer readable medium, and processor each only amount to mere instructions to implement the abstract idea using computing elements as tools. For example, the processor is only recited at a high level of generality as performing the subsequent functions and disclosed broadly as a data processor, while the non-transitory computer readable medium is likewise recited at a high level of generality as storing computer readable instructions and disclosed as generic forms of computer memory. These elements are therefore not sufficient to integrate the abstract idea into a practical application.
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
Claims 1, 14, and 15 recite a further additional element of receiving the data collected during the clinical procedure. This element only amounts to insignificant extra-solution activity in the form of data gathering since performing the abstract idea requires receiving the data collected during the clinical procedure. This element is therefore not sufficient to integrate the abstract idea into a practical application.
The above claims, as a whole, are therefore directed to an abstract idea.
Step 2B
The present claims do not include additional elements that are sufficient to amount to more than the abstract idea because the additional elements or combination of elements amount to no more than a recitation of:
A. Instructions to Implement the Judicial Exception. MPEP 2106.05(f)
As explained above, claims 1 and 15 only recite the processing circuitry, non-transitory computer readable medium, and processor as tools for performing the steps of the abstract idea, and mere instructions to perform the abstract idea using a computer is not sufficient to amount to significantly more than the abstract idea. MPEP 2106.05(f)
B. Insignificant Extra-Solution Activity. MPEP 2106.05(g)
As addressed above, claims 1, 14, and 15 recite further recite the element of receiving the data collected during the clinical procedure. This element only amounts to insignificant extra-solution activity in the form of data gathering since performing the abstract idea requires receiving the data collected during the clinical procedure.
C. Well-Understood, Routine and Conventional Activities. MPEP 2106.05(d)
The element of receiving the data collected during the clinical procedure additionally only amounts to well-understood routine and conventional activity, such as in the form of receiving or transmitting data over a network. See MPEP 2106.05(d)(II).
Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
Depending Claims
Claim 2 recites wherein for each of the plurality of data modalities, the step of processing the labels for each of the identified events comprises processing the labels to obtain the output indicative of the further medical event. These limitations fall within the scope of the abstract idea as set out above.
Claim 2 further recites the additional element of a machine learning model used to process the labels to obtain the output indicative of the further medical event.
Page 13 lines 5-6 and page 18 lines 5-9 describe processing the labels using a machine learning model, such as a recurrent neural network.
The recited machine learning model only amounts to mere instructions to implement the abstract idea using computing elements as tools. The machine learning model is only recited at a high level of generality as receiving the labels as inputs in order to obtain the output indicative of the further medical event and disclosed broadly as encompassing architectures such as a recurrent neural network. The machine learning model is only operating in its generic capacity as a mechanism to process the label data, and is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 3 recites wherein for each of the data modalities, the processing of the data of the respective data modality and the generation of the label is performed in real time as the data is received, for each of the labels, upon the generation of the respective label, processing the labels. These limitations fall within the scope of the abstract idea as set out above.
Claim 3 further recites the additional element of the machine learning model as being provided with the labels as inputs and processing the labels.
Page 13 lines 5-6 and page 18 lines 5-9 describe processing the labels using a machine learning model, such as a recurrent neural network.
The recited machine learning model only amounts to mere instructions to implement the abstract idea using computing elements as tools. The machine learning model is only recited at a high level of generality as receiving the labels as inputs in order to obtain the output indicative of the further medical event and disclosed broadly as encompassing architectures such as a recurrent neural network. The machine learning model is only operating in its generic capacity as a mechanism to process the label data, and is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 4 recites the additional element of the machine learning model being a recurrent neural network.
Page 13 lines 5-6 and page 18 lines 5-9 describe processing the labels using a recurrent neural network, which receives labels as inputs and provides an output representing an event.
The recited recurrent neural network only amounts to mere instructions to implement the abstract idea using computing elements as tools. The recurrent neural network is only recited at a high level of generality as receiving the labels as inputs in order to obtain the output indicative of the further medical event and disclosed broadly. The recurrent neural network is only operating in its generic capacity as a mechanism to process the label data, and is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 5 recites provide each of the labels for processing in an order in which the corresponding identified events occurred in the common timeline. These limitations fall within the scope of the abstract idea as set out above.
Claim 5 further recites the additional elements of a) the processing circuitry as performing the subsequent data processing functions, and b) the machine learning model as receiving the labels as inputs and processing the labels.
Page 4 line 23 – page 5 line 32 describe an apparatus for performing the disclosed functions, where the apparatus includes a user mobile device, a personal computer, a server, or “some other form of device.” The apparatus is disclosed as having a data processor as well as non-transitory memory devices storing computer readable instructions for execution by the data processor. The processing circuitry is therefore construed as encompassing generic computing components.
Page 13 lines 5-6 and page 18 lines 5-9 describe processing the labels using a machine learning model, such as a recurrent neural network.
These elements only amount to mere instructions to implement the abstract idea using computing elements as tools. The processing circuitry is only recited at a high level of generality as performing the subsequent functions and is disclosed broadly as a data processor. The machine learning model is likewise only recited at a high level of generality as receiving the labels as inputs in order to obtain the output indicative of the further medical event and is only operating in its generic capacity as a mechanism to process the label data These elements are therefore not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than an abstract idea.
Claim 6 recites for each of the identified events, outputting time information indicating a time in the common timeline at which the respective identified event occurred; and processing the time information for the identified events to determine a time associated with the further medical event. These limitations fall within the scope of the abstract idea as set out above.
Claim 6 further recites the additional elements of a) the processing circuitry as performing the subsequent data processing functions.
Page 4 line 23 – page 5 line 32 describe an apparatus for performing the disclosed functions, where the apparatus includes a user mobile device, a personal computer, a server, or “some other form of device.” The apparatus is disclosed as having a data processor as well as non-transitory memory devices storing computer readable instructions for execution by the data processor. The processing circuitry is therefore construed as encompassing generic computing components.
The processing circuitry only amounts to mere instructions to implement the abstract idea using computing elements as tools. The processing circuitry is only recited at a high level of generality as performing the subsequent functions and is disclosed broadly as a data processor. This element is therefore not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than an abstract idea.
Claim 7 recites wherein the imaging data comprises at least one of: video data; and medical imaging data. These limitations fall within the scope of the abstract idea as set out above.
Claim 8 recites wherein the plurality of data modalities comprises one or more of: video data: audio data; medical imaging data; or radio frequency tag data. These limitations fall within the scope of the abstract idea as set out above.
Claim 9 recites wherein for one or more of the plurality of data modalities: the processing of the data of the respective data modality to identify the event occurring during the procedure comprises processing the data of the respective data modality to derive the label of the respective identified event. These limitations fall within the scope of the abstract idea as set out above.
Claim 9 further recites the additional element of a further machine learning model used to process the data of the respective data modality to derive the label.
Page 16 lines 27-33 and page 24 lines 33-36 describe processing modality data using a machine learning model, such as a convolutional neural network.
The recited further machine learning model only amounts to mere instructions to implement the abstract idea using computing elements as tools. The further machine learning model is only recited at a high level of generality as receiving the respective modality data to derive the labels, and disclosed broadly as encompassing architectures such as a convolutional neural network. The machine learning model is only operating in its generic capacity as a mechanism to process the modality data, and is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 10 recites wherein one or more of the plurality of data modalities comprises the imaging data, and processing the imaging data to derive the label of the respective identified event. These limitations fall within the scope of the abstract idea as set out above.
Claim 10 further recites the additional element of the further machine learning model as used to process the imaging data and as comprising a convolutional neural network.
Page 11 lines 8-10, page 16 lines 27-33 and page 24 lines 33-36 describe processing modality data using a machine learning model in the form of a convolutional neural network.
The recited convolutional neural network only amounts to mere instructions to implement the abstract idea using computing elements as tools. The convolutional neural network is only recited at a high level of generality as receiving the respective modality data to derive the labels, and is only operating in its generic capacity as a mechanism to process the modality data. This element is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 11 recites wherein one or more of the plurality of data modalities comprises audio data, wherein the audio data is processed to derive text representing the audio data, and to derive the label of the respective identified event. These limitations fall within the scope of the abstract idea as set out above.
Claim 11 further recites the additional elements of the further machine learning model as used to process the audio data and as comprising a speech recognition model configured to derive text representing the audio data.
Page 9 lines 8-11 and page 10 lines 12-15 describe the use of a speech recognition model to derive text from received audio data. No further disclosure of a specific speech recognition algorithm is provided.
The recited speech recognition model only amounts to mere instructions to implement the abstract idea using computing elements as tools. The speech recognition model is only recited at a high level of generality as processing the audio data to derive text, and is only operating in its generic capacity as a mechanism to convert the audio data to text. This element is not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 12 recites processing the text to derive the label identifying the event. These limitations fall within the scope of the abstract idea as set out above.
Claim 12 further recites the additional elements of a) the processing circuitry as performing the subsequent data processing functions and b) a natural language understanding model as used to process the text to derive the label.
Page 4 line 23 – page 5 line 32 describe an apparatus for performing the disclosed functions, where the apparatus includes a user mobile device, a personal computer, a server, or “some other form of device.” The apparatus is disclosed as having a data processor as well as non-transitory memory devices storing computer readable instructions for execution by the data processor. The processing circuitry is therefore construed as encompassing generic computing components.
Page 9 lines 8-11 and page 10 lines 12-15 describe the use of a natural language understanding model to derive a label from the text. No further disclosure of a natural language understanding model is provided.
These elements only amount to mere instructions to implement the abstract idea using computing elements as tools. The processing circuitry is only recited at a high level of generality as performing the subsequent functions and is disclosed broadly as a data processor. The natural language understanding model is likewise only recited at a high level of generality as processing the text data to derive a label, and is only disclosed broadly. These elements are not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claim 13 recites determining the further medical event and associated time information indicating when, in the clinical procedure, the further medical event took place. These limitations fall within the scope of the abstract idea as set out above.
Claim 13 further recites the additional elements of a) the processing circuitry as controlling a display, b) the display recited as providing a visual display indicative of the further medical event and associated time information.
Page 4 line 23 – page 5 line 32 describe an apparatus for performing the disclosed functions, where the apparatus includes a user mobile device, a personal computer, a server, or “some other form of device.” The apparatus is disclosed as having a data processor as well as a display for displaying visual content. Page 22 lines 10-13 also describes a display provided on a user interface including a timeline of events. The processing circuitry and display are construed as encompassing generic computing components, while the visual display is similarly construed as encompassing graphical elements on a computer interface.
These elements only amount to mere instructions to implement the abstract idea using computing elements as tools. The processing circuitry is only recited at a high level of generality as performing the subsequent functions and is disclosed broadly as a data processor. The display and visual display are likewise only recited at a high level of generality as providing a visual display of the generated medical event and time information, and are likewise only disclosed broadly. These elements are not sufficient to integrate the abstract idea into a practical application or to amount to significantly more than the abstract idea.
Claims 1-15 are therefore rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 7-11, 14, and 15 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Venkataraman et al (US Patent Application Publication 2022/0028525).
With respect to claim 1, Venkataraman discloses the claimed data processing apparatus (Figure 4 and [60]-[62] describe a computing device performing the functions) comprising processing circuitry configured to:
receive data collected during a clinical procedure, the data belonging to a plurality of data modalities, at least one of the data modalities being an imaging data type and a further of the data modalities being an additional data type other than imaging data (Figure 1, Figure 2 element 202, [33], [40], [47], and [53] describe the system receiving data collected during a surgical procedure from a plurality of modalities including audio data, image data, and video data), wherein the data is provided with reference to a common timeline over which the data is collected ([8], [50], and [58] describe the data from each of the modalities being time synchronized based on common timestamps);
for each of the plurality of data modalities, process data of the respective data modality to generate one or more labels, each identifying an event occurring at a time on the common timeline and indicated by the processed data (Figure 1 elements 102-106, Figure 2 element 204, [31], [32], [37], [38], and [41]-[45] describe a set of segmentation engines extracting information from the modality data and converting it into a text representation of the data contents, i.e. a label. Examiner notes page 8 lines 13-29 of Applicant’s specification as-filed, describing labels as numerical or string values representing information such as contents of an image or pieces of medical equipment); and
process the labels for each of the identified events based on the relative time of occurrence of the events to obtain an output indicative of a further medical event ([32], [46], [49], [50], [56], and [59] describe combining the text features and temporal data into a comprehensive feature set and determining further events including successful or unsuccessful removal of tissue, outcomes, and the presence of particular tissues in real time).
With respect to claim 2, Venkataraman discloses the data processing apparatus as claimed in claim 1. Venkataraman further discloses:
wherein for each of the plurality of data modalities, the step of processing the labels for each of the identified events comprises: providing the labels as inputs to a machine learning model to obtain the output indicative of the further medical event ([49] describes using a machine learning model to process the combined text features and output information such as a location of a tumor or whether the tumor has been correctly identified and removed).
With respect to claim 3, Venkataraman discloses the data processing apparatus as claimed in claim 2. Venkataraman further discloses:
wherein for each of the data modalities, the processing of the data of the respective data modality and the generation of the label is performed in real time as the data is received, wherein providing the labels as inputs to the machine learning model comprises, for each of the labels, upon the generation of the respective label: providing the respective label as an input to the machine learning model (Claim 5, [12], [32], [40], and [59] describe the data being collected in real time during the procedure as well as performing the data analytics in real time during the procedure, where the data analytics is described as including the analysis of the combined feature set).
With respect to claim 7, Venkataraman discloses the data processing apparatus as claimed in claim 1. Venkataraman further discloses:
wherein the imaging data comprises at least one of video data and medical imaging data (Figure 1 elements 112 and 114, [10], [11], and [40] describe the data including both video data from cameras and medical imaging data from various patient imaging modalities).
With respect to claim 8, Venkataraman discloses the data processing apparatus as claimed in claim 1. Venkataraman further discloses:
wherein the plurality of data modalities comprises one or more of video data, audio data, medical imaging data (Figure 1 elements 112-116, [10]-[12], and [40] describe the data including audio data, video data from cameras, and medical imaging data from various patient imaging modalities), or radio frequency tag data.
With respect to claim 9, Venkataraman discloses the data processing apparatus as claimed in claim l. Venkataraman further discloses:
wherein for one or more of the plurality of data modalities: the processing of the data of the respective data modality to identify the event occurring during the procedure comprises providing the data of the respective data modality to a further machine learning model to derive the label of the respective identified event ([31], [41], and [43] describe the segmentation engines used to derive the text features as including machine learning models).
With respect to claim 10, Venkataraman discloses the data processing apparatus as claimed in claim 9. Venkataraman further discloses:
wherein one or more of the plurality of data modalities comprises the imaging data ([10], [11], and [40] describe the data including medical imaging data from various patient imaging modalities),
wherein, for the imaging data, the respective further machine learning model used to derive the label of the respective identified event comprises a convolutional neural network ([43] describes each of the segmentation machine learning models as constructed based on a convolutional neural network).
With respect to claim 11, Venkataraman discloses the data processing apparatus as claimed in claim 9. Venkataraman further discloses:
wherein one or more of the plurality of data modalities comprises audio data (Figure 1 element 116, [12], and [40] describe the data including audio data),
wherein for the audio data, the respective further machine learning model used to derive the label of the respective identified event comprises a speech recognition model configured to derive text representing the audio data ([31], [33], [38], [41] and [43] describe using a machine learning model to convert a surgeon’s narration and comments into text data).
With respect to claim 14, Venkataraman discloses the method comprising:
receiving data collected during a clinical procedure, the data belonging to a plurality of data modalities, at least one of the data modalities being an imaging data type and a further of the data modalities being an additional data type other than imaging data (Figure 1, Figure 2 element 202, [33], [40], [47], and [53] describe the system receiving data collected during a surgical procedure from a plurality of modalities including audio data, image data, and video data), wherein the data is provided with reference to a common timeline over which the data is collected ([8], [50], and [58] describe the data from each of the modalities being time synchronized based on common timestamps);
for each of the plurality of data modalities, processing data of the respective data modality to generate a label identifying an event occurring at a time on the common timeline and indicated by the processed data (Figure 1 elements 102-106, Figure 2 element 204, [31], [32], [37], [38], and [41]-[45] describe a set of segmentation engines extracting information from the modality data and converting it into a text representation of the data contents, i.e. a label. Examiner notes page 8 lines 13-29 of Applicant’s specification as-filed, describing labels as numerical or string values representing information such as contents of an image or pieces of medical equipment); and
processing the labels for each of the identified events based on the relative time of occurrence of the events to obtain an output indicative of a further medical event ([32], [46], [49], [50], [56], and [59] describe combining the text features and temporal data into a comprehensive feature set and determining further events including successful or unsuccessful removal of tissue, outcomes, and the presence of particular tissues in real time).
With respect to claim 15, Venkataraman discloses the non-transitory computer-readable medium storing a computer program comprising computer readable computer-readable instructions, which when executed by at least one processor, causes the at least one processor to (Figure 4, [60]-[62], and [70] describe a computing device having a processor and non-transitory memory as performing the functions) perform a method comprising:
receiving data collected during a clinical procedure, the data belonging to a plurality of data modalities, at least one of the data modalities being an imaging data type and a further of the data modalities being an additional data type other than imaging data (Figure 1, Figure 2 element 202, [33], [40], [47], and [53] describe the system receiving data collected during a surgical procedure from a plurality of modalities including audio data, image data, and video data), wherein the data is provided with reference to a common timeline over which the data is collected ([8], [50], and [58] describe the data from each of the modalities being time synchronized based on common timestamps);
for each of the plurality of data modalities, processing data of the respective data modality to generate a label identifying an event occurring at a time on the common timeline and indicated by the processed data (Figure 1 elements 102-106, Figure 2 element 204, [31], [32], [37], [38], and [41]-[45] describe a set of segmentation engines extracting information from the modality data and converting it into a text representation of the data contents, i.e. a label. Examiner notes page 8 lines 13-29 of Applicant’s specification as-filed, describing labels as numerical or string values representing information such as contents of an image or pieces of medical equipment); and
processing the labels for each of the identified events based on the relative time of occurrence of the events to obtain an output indicative of a further medical event ([32], [46], [49], [50], [56], and [59] describe combining the text features and temporal data into a comprehensive feature set and determining further events including successful or unsuccessful removal of tissue, outcomes, and the presence of particular tissues in real time).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Venkataraman et al (US Patent Application Publication 2022/0028525) as applied to claim 2 above, and further in view of Jin et al (US Patent Application Publication 2020/0226751).
With respect to claim 4, Venkataraman discloses the data processing apparatus as claimed in claim 2. Venkataraman does not expressly disclose wherein the machine learning model is a recurrent neural network.
However, Jin teaches that it was old and well known in the art of surgical monitoring before the effective filing date of the claimed invention to process labels from imaging data using a recurrent neural network to obtain a further medical event (Figure 3C, [45], and [70]-[72] describe extracting features from frames and inputting them to a recurrent neural network to obtain a phase of the surgery).
Therefore it would have been obvious to one of ordinary skill in the art of surgical monitoring before the effective filing date of the claimed invention to modify the system of Venkataraman to process labels to obtain a further medical event using a recurrent neural network as taught by Jin since the claimed invention is only a combination of these old and well known elements which would have performed the same function in combination as each did separately. In the present case Venkataraman already discloses providing labels as inputs to a machine learning model to obtain the output of the further medical event (see citations above with respect to claim 2), and using a recurrent neural network as that machine learning model as taught by Jin would serve that same function in Venkataraman, making the results predictable to one of ordinary skill in the art (MPEP 2143).
Claims 5, 6, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Venkataraman et al (US Patent Application Publication 20220028525) as applied to claim 2 above, and further in view of Akiyama et al (US Patent Application Publication 2022/0031271).
With respect to claim 5, Venkataraman discloses the data processing apparatus as claimed in claim 2, wherein the processing circuitry is further configured to:
provide each of the labels as inputs to the machine learning model based on when they occurred in the common timeline ([50], [51], and [58] describe determining a set of labels occurring in a common portion of the timeline and grouping them for processing; [49] describes using a machine learning model to process combined text features);
but does not expressly disclose:
providing the labels as inputs in an order in which the corresponding identified events occurred.
However, Akiyama teaches that it was old and well known in the art of surgical monitoring before the effective filing date of the claimed invention to process labels in an order in which the corresponding identified events occurred ([33], [64], [68]-[70], and [72] describe sequentially acquiring medical images during a procedure and providing each image in sequence to an event detector).
Therefore it would have been obvious to one of ordinary skill in the art of surgical monitoring before the effective filing date of the claimed invention to modify the system of Venkataraman to process labels to process the labels in an order in which the corresponding identified events occurred as taught by Akiyama since the claimed invention is only a combination of these old and well known elements which would have performed the same function in combination as each did separately. In the present case Venkataraman already discloses providing the labels to the machine learning model based on when the corresponding identified events occurred, and doing so in an order in which they occurred as taught by Akiyama would serve that same function in Venkataraman, making the results predictable to one of ordinary skill in the art (MPEP 2143).
With respect to claim 6, Venkataraman discloses the data processing apparatus as claimed in claim 1, wherein the processing circuitry is further configured to:
for each of the identified events, output time information indicating a time in the common timeline at which the respective identified event occurred ([50] describes the system associating a timestamp with identified events such as discussion of a tumor in the audio data); and
process the time information for the identified events ([50] describes processing timestamps for the events);
but does not expressly disclose:
processing the time information to determine a time associated with the further medical event.
However, Akiyama teaches that it was old and well known in the art of surgical monitoring before the effective filing date of the claimed invention to use time information associated with events to determine a time associated with a further medical event (Figures 5 and 12, [68]-[70], [85], [89], and [90] describe recording the time corresponding to each medical images during a procedure and using events in the image, such as positions of tools, to determine the time of a further associated event such as ‘balloon dilation’).
Therefore it would have been obvious to one of ordinary skill in the art of surgical monitoring before the effective filing date of the claimed invention to modify the system of Venkataraman to use time information associated with events to determine a time associated with a further medical event as taught by Akiyama since the claimed invention is only a combination of these old and well known elements which would have performed the same function in combination as each did separately. In the present case Venkataraman already discloses identifying a time for identified events and processing that time information, and doing so to determine a time associated with the further medical event as taught by Akiyama would serve that same function in Venkataraman, making the results predictable to one of ordinary skill in the art (MPEP 2143).
With respect to claim 13, Venkataraman discloses the data processing apparatus as claimed in claim 1. Venkataraman further discloses
wherein the processing circuitry is further configured to: control a display to provide a visual display indicative of the further medical event ([46] describes outputting visualizations of the metrics and outcomes);
but does not expressly disclose:
the visual display indicating associated time information indicating when, in the clinical procedure, the further medical event took place.
However, Akiyama teaches that it was old and well known in the art of surgical monitoring before the effective filing date of the claimed invention to provide a visual display indicating associated time information indicating when, in a clinical procedure, a further medical event took place (Figure 12, [89]-[92], [95], [126], and [127] describe outputting a timeline showing the chronological order and time of each image and further event).
Therefore it would have been obvious to one of ordinary skill in the art of surgical monitoring before the effective filing date of the claimed invention to modify the system of Venkataraman to provide a visual display indicating associated time information indicating when, in a clinical procedure, a further medical event took place as taught by Akiyama since the claimed invention is only a combination of these old and well known elements which would have performed the same function in combination as each did separately. In the present case Venkataraman already discloses providing a visual display indicative of the further medical event, and having the display indicate associated time information indicating when, in the clinical procedure, the further medical event took place as taught by Akiyama would serve that same function in Venkataraman, making the results predictable to one of ordinary skill in the art (MPEP 2143).
Claims 12 is rejected under 35 U.S.C. 103 as being unpatentable over Venkataraman et al (US Patent Application Publication 20220028525) as applied to claim 11 above, and further in view of Guzman-Garcia et al Speech-Based Surgical Phase Recognition for Non-Intrusive Surgical Skills’ Assessment in Educational Contexts (hereinafter Guzman-Garcia).
With respect to claim 12, Venkataraman discloses the data processing apparatus as claimed in claim 11. Venkataraman does not expressly disclose processing the text using a natural language understanding model to derive the label identifying the event
However, Guzman-Garcia teaches that it was old and well known in the art of surgical monitoring before the effective filing date of the claimed invention to process text using a natural language understanding model to derive a label identifying a medical event (§§1.3 and 2.2 describe transcribing audio of a procedure into text and then further applying natural language processing algorithms to derive context and meaning for features).
Therefore it would have been obvious to one of ordinary skill in the art of surgical monitoring before the effective filing date of the claimed invention to modify the system of Venkataraman to process the text using a natural language understanding model to derive the label identifying a medical event as taught by Guzman-Garcia since the claimed invention is only a combination of these old and well-known elements which would have performed the same function in combination as each did separately. In the present case Venkataraman already discloses transcribing audio data into text, and processing that text using a natural language understanding model to derive the label as taught by Guzman-Garcia would serve that same function in Venkataraman, making the results predictable to one of ordinary skill in the art (MPEP 2143).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dhariwal et al (US Patent Application Publication 2023/0326172);
Mitrea et al (US Patent Application Publication 2024/0136045);
Luengo Muntion et al US Patent Application Publication 2024/0161497).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM G LULTSCHIK whose telephone number is (571)272-3780. The examiner can normally be reached 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long can be reached at (571) 270-5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Gregory Lultschik/Examiner, Art Unit 3682