DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Concerning Applicant’s argument relating to the drawings, the Examiner apologizes on the oversite. The Figures show boxes without labelling the function/step performed by each box.
Concerning Applicant’s argument relating to 112 6th claim interpretation, the Examiner disagrees. Applicant alleges a POSITA would understand the term to have a sufficiently definite meaning, but fails to define that actual structure. A filter, brake, claim , screwdriver and lock all have some type of definite structure. For example, I have a screwdriver, everyone can envision what the structure of a screwdriver can look like, no matter if it is a flat head, Phillips head, or electric screw driver. What is the actual structure of a signal processing device?
The 112 2nd rejections have been withdrawn, with the exception of claim 13. Claim 13 line 2 recites “the compression of the modified video image data”. The compression lacks antecedent basis. As well as line 4, “the selected compression method”.
Applicant argues
PNG
media_image1.png
186
588
media_image1.png
Greyscale
The Examiner disagrees. The cited paragraphs explicitly states “At the time at which the at least one element of the time-variant process data is detected, a source image is generated by the camera at the same time or in the immediate temporal context. The term temporally immediate is to be understood in particular as meaning that a first time at which the source image is generated and a second time at which the at least one element of the time-variant process data is detected by at most one second, preferably at most 0.3 Seconds, more preferably at most 0.1 second and in particular at most 0.05 seconds apart. In a preferred embodiment, the first and second times are at most 0.016 seconds apart. The term "time-synchronized" as used herein should be understood in the sense of this immediate temporal context, in particular as being simultaneous.”. The it is clear the time-variant data is acquired up to 0.016 seconds apart. The cited paragraph does not discuss synchronization. Even if Bruderle discussed acquiring ONLY one time variant data per frame, and not multiple (which is what the Examiner is assuming applicant is arguing), Fig. 4 clearly shows multiple data being acquired such as Blood pressure, heart rate as well as a waveform which comprises a plurality of values.
Applicant argues
PNG
media_image2.png
374
624
media_image2.png
Greyscale
The examiner does contend it is common knowledge to a POSITA that ECG frequencies typically range in the 0 to 250 Hz ranges, for adults 0 to 150, generally.
For example, the 2007 paper (“Kligfield, P, Gettes, L, Bailey, J. et al. Recommendations for the Standardization and Interpretation of the Electrocardiogram: Part I: The Electrocardiogram and Its Technology A Scientific Statement From the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society Endorsed by the International Society for Computerized Electrocardiology. JACC. 2007 Mar, 49 (10) 1109–1127. https://doi.org/10.1016/j.jacc.2007.01.024”) states
PNG
media_image3.png
382
752
media_image3.png
Greyscale
And the 2015 paper(Ellis, Robert J. et al. “A careful look at ECG sampling frequency and R-peak interpolation on short-term measures of heart rate variability.” Physiological Measurement 36 (2015): 1827 - 1852.) states
PNG
media_image4.png
300
514
media_image4.png
Greyscale
Even if it were not the case, The prior art shows display multiple measured values (BP, HEART RATE & ECG) data thus meeting the limitation. And the waveform alone shows a plurality of measured values.
Drawings
The drawings are objected to because Fig. 1, 2 and 5 show boxes without text descriptors for each block. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “medical device” and “signal processing device” in claim 17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 13 recites “the compression of the modified video image data”. This limitation lacks antecedent basis.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bruderle(English Translation of DE102017105169A1)
Bruderle discloses 1. A method for supporting evaluation of a video-assisted medical interventional procedure, the method comprising:
receiving medical video image data representing video images of an examined or treated anatomy recorded by a video camera at a specific frame rate during a medical intervention; (Bruderle, pg. 2 “In preferred embodiments, the dynamic vital parameters of the patient, the dynamic process data that arise during an operation at a time, continuously and inseparably connected to the simultaneously generated, in particular in real time, source images, especially video images. The real-time images, in particular in the form of a video, are produced at the respective time and can be endoscopic images, images from documentation cameras or images from spatial cameras..”, all videos captured are at a user-specified or device specific frame rate)
receiving further data comprising at least one of treatment or examination, device, diagnostic and measurement data associated with the medical intervention, the further data including measured dynamic data that varies during a frame period; (Bruderle, pg. 2 “In preferred embodiments, the dynamic vital parameters of the patient, the dynamic process data that arise during an operation at a time, continuously and inseparably connected to the simultaneously generated, in particular in real time, source images, especially video images. The real-time images, in particular in the form of a video, are produced at the respective time and can be endoscopic images, images from documentation cameras or images from spatial cameras..”, “At the time at which the at least one element of the time-variant process data is detected, a source image is generated by the camera at the same time or in the immediate temporal context. The term temporally immediate is to be understood in particular as meaning that a first time at which the source image is generated and a second time at which the at least one element of the time-variant process data is detected by at most one second, preferably at most 0.3 Seconds, more preferably at most 0.1 second and in particular at most 0.05 seconds apart. In a preferred embodiment, the first and second times are at most 0.016 seconds apart. The term "time-synchronized" as used herein should be understood in the sense of this immediate temporal context, in particular as being simultaneous.”)
identifying target areas in the video images for embedding the further data; and (Bruderle, pg. 3, “This embodiment makes it possible in a simple way to reduce the possibility that the code covers relevant information from the source image in the result image. It is preferred that the code is directly adjacent to the edge region of the source image. Furthermore, it is preferred that the code is arranged in a corner region of the source image, in particular directly adjacent to the edges of a corner.”)
modifying the video image data by embedding the further data into the video image data synchronously in time by replacing video image data in the identified target area or areas of a frame with the further data associated with the respective frame. (Bruderle, pg. 2, “In an image processing unit, the source image from the camera is combined with the computer-readable code. The combination takes place in such a way that both the source image and the code can be seen in a result image. As will be explained below, it is not necessary that every single pixel of the source image is also included in the result image. Rather, it is sufficient that a person can recognize relevant information contained in the source image also in the result image. This can technically be achieved in preferred embodiments in that the code covers at most 25%, preferably at most 10%, particularly preferably at most 5% and in particular at most 3% of the area of the source image.”, where the code represents the process data), wherein wherein the embedded further data associated with the respective frame includes multiple measured values of the measured dynamic data during the frame period.(Bruderle, Fig. 4, se Pulse, Blood pressure, and chart below)
Bruderle discloses 2. The method of claim 1, further comprising at least one of the following steps:
storing the modified video image data in a non-volatile storage device for further use; and (Bruderle, pg. 2, “In this way, it can be ensured, in particular, that the current time-variant process data are stored in a computer-readable manner synchronously with a corresponding source image at all times and can be permanently archived.”)
transmitting the modified video image data to a PACS picture archiving and communication system, a CIS/HIS hospital information system or an external server for archiving image data.(Bruderle, Figure 1)
Bruderle discloses 3. The method according to claim 1, wherein the frame rate is between 24 and 60 frames per second.(Bruderle discloses a generic video camera, but doesn’t explicitly disclose the frame rate; The video standard video frame rate is 30fps; The standard cinematic standard video frame rate is 24 fps. The slow motion standard frame rate is 60fps (see for example https://wistia.com/learn/production/what-is-frame-rate), therefore the generic video camera encompasses the frame rate of 24 to 60 )
Bruderle discloses 4. The method according to claim 1, wherein the treatment or examination data include place and time of the intervention, institution involved, persons involved, name of the patient, name of the attending physician, type of medical intervention or other data relating to the medical intervention; (Bruderle, pg. 2, “In an advantageous embodiment, the process data comprise at least one element from the group consisting of a vital parameter of the patient, a setting parameter of a medical device, a measured value of a medical device, information about the operation on the patient and information for handling surgical instruments.”)
wherein the device data includes a type designation of the medical device or devices used, device serial number, device parameters, default settings, selected application mode, preset limit values for variables, error and status messages from the devices or other data relating to the medical devices used; wherein the diagnostic data includes determined spectra or classification results from an optical emission spectrometry, spectra, measured values and/or classification results from an impedance spectroscopy or from other diagnostic methods; (See citation above)
wherein the measurement data includes measured or determined electrical RF variables, such as voltage, current, power, power factor and/or spark intensity, parameters or variables of a neutral electrode, such as transition impedance, current symmetry and/or current density, or other measurement variables measured or determined during the medical intervention. (See citation above)
Bruderle discloses 5. The method according to claim 1, wherein the measured dynamic data includes measurement variables that are acquired or determined at an update rate that is several times greater than the frame rate of the video image data, and the update rate is at least 150 Hz.(Bruderle, pg. 2 “At the time at which the at least one element of the time-variant process data is detected, a source image is generated by the camera at the same time or in the immediate temporal context. The term temporally immediate is to be understood in particular as meaning that a first time at which the source image is generated and a second time at which the at least one element of the time-variant process data is detected by at most one second, preferably at most 0.3 Seconds, more preferably at most 0.1 second and in particular at most 0.05 seconds apart. In a preferred embodiment, the first and second times are at most 0.016 seconds apart. The term "time-synchronized" as used herein should be understood in the sense of this immediate temporal context, in particular as being simultaneous.”; Additionally Bruderle discloses a heart rate monitor in an operating room setting. ECG frequencies typically range in the 150 Hz speed, see for example https://www.gehealthcare.com/insights/article/a-guide-to-ecg-signal-filtering)
Bruderle discloses 6. The method according to claim 1, wherein measured dynamic data which characterizes a time course of two or more measured values of a measured electrical variable, is stored in individual frames. (Bruderle, pg. 2-3, “Among the time-variant process data, in preferred embodiments at least one element from the group consisting of vital parameters of the patient during the operation, least one element from the group consisting of heart rate, blood pressure, body temperature and respiratory rate. The medical device data include in particular at least one element selected from the group consisting of serial numbers and versions of the medical devices used, anesthetic data, settings of the medical devices and surgical sieves used. Information about the current stage of the operation includes, in particular, the currently performed surgical procedure and / or the current listing of the surgical team involved in the procedure.
The anesthesia data are, in particular, at least one element selected from the group comprising the setpoint frequency of the ventilation, the actual frequency of the ventilation, the setpoint volume of the ventilation and the actual volume of the ventilation. The data of the medical devices are, in particular, at least one desired value and/ or actual value of at least one parameter of at least one medical device selected from the group consisting of pump, insufflator, light source, camera, laser device and operating table. In particular, at least one access or exit of at least one surgical instrument is selected when using the surgical screens of the group consisting of pliers, tweezers, scissors, hooks and clamps.”)
Bruderle discloses 7. The method according to claim 1, wherein identifying target areas in the video images for embedding the further data is based on areas in the video images that are defined in advance or are specified or can be specified by a user. (Bruderle, pg. 3, “This embodiment makes it possible in a simple way to reduce the possibility that the code covers relevant information from the source image in the result image. It is preferred that the code is directly adjacent to the edge region of the source image. Furthermore, it is preferred that the code is arranged in a corner region of the source image, in particular directly adjacent to the edges of a corner.”)
Bruderle discloses 8. The method according to claim 1, wherein the further data is stored at defined positions distributed over the image. (Bruderle, pg. 3, “This embodiment makes it possible in a simple way to reduce the possibility that the code covers relevant information from the source image in the result image. It is preferred that the code is directly adjacent to the edge region of the source image. Furthermore, it is preferred that the code is arranged in a corner region of the source image, in particular directly adjacent to the edges of a corner.”)
Bruderle discloses 9. The method according to claim 1, wherein the further data is embedded in areas of the video images for labeling, displayed patient data, time and/or at an image edge. (Bruderle, pg. 3, “This embodiment makes it possible in a simple way to reduce the possibility that the code covers relevant information from the source image in the result image. It is preferred that the code is directly adjacent to the edge region of the source image. Furthermore, it is preferred that the code is arranged in a corner region of the source image, in particular directly adjacent to the edges of a corner.”)
Bruderle discloses 10. The method according to claim 1, wherein the further data is incorporated into individual bits of a pixel color of a plurality of pixels of the image data.(Bruderle, pg. “In a further advantageous embodiment, the code is selected from a group consisting of a 2D barcode, a 3D barcode, a 4D barcode, a QR code, a QR color code and an recode.”)
Bruderle discloses 11. The method according to claim 10, wherein the further data is incorporated in image areas with a lower useful information content. (Bruderle, pg. 3, “This embodiment makes it possible in a simple way to reduce the possibility that the code covers relevant information from the source image in the result image. It is preferred that the code is directly adjacent to the edge region of the source image. Furthermore, it is preferred that the code is arranged in a corner region of the source image, in particular directly adjacent to the edges of a corner.”)
Bruderle discloses 12. The method according to claim 1, wherein the modified video image data is lossy compressed, wherein the modified video image data is processed by a compression invariance enhancement procedure before a compression method is performed, in order to increase reliability of a reconstruction of the further data.(Bruderle, pg. 3 “The display device offers an advantageous possibility of making available the process data embedded as code in the result image, in particular a video formed from a plurality of result images. Since the process data are stored time-synchronously with a respective result image, a time-synchronized representation of process data and corresponding source images or result images is possible even after years or decades. The representation of the result images or source images contained in several result images also includes the possibility of displaying the result images or source images in a different manner, e.g. fragmentary, enlarged, reduced, cropped, compressed, with altered resolution etc.”)
Bruderle discloses 13. The method according to claim 1, wherein the modification of the video image data and a compression of the modified video image data are performed in a common signal processing device, (Bruderle, pg. 5 “In an advantageous embodiment, the coded, optionally encrypted information is stored as a subtitle in the result image, in particular stored frame-accurate in a video image. In a further preferred embodiment, the result images form a 2D video or a 3D video. In a further advantageous embodiment, the vital parameters of the patient and the data of the medical devices are transmitted to the data acquisition unit via wired bus systems. In another advantageous embodiment, the vital parameters of the patient and the data of the medical devices are wirelessly transmitted to the data acquisition unit. In a further preferred refinement, a processor of the image processing unit is designed both to generate the result image and to generate the coded encrypted process data and to combine both in a time-synchronized manner. In another advantageous embodiment, a second processor for generating the encoded encrypted process data is connected downstream of the first processor for generating the result image. In a further advantageous embodiment, a total checksum over all result images, i.e. over an entire video, is formed at the end of a recording over all checksums of the individual result images, in particular video frames.”) which performs the compression invariance enhancement 1procedure depending on the selected compression method. .(Bruderle, pg. 3 “The display device offers an advantageous possibility of making available the process data embedded as code in the result image, in particular a video formed from a plurality of result images. Since the process data are stored time-synchronously with a respective result image, a time-synchronized representation of process data and corresponding source images or result images is possible even after years or decades. The representation of the result images or source images contained in several result images also includes the possibility of displaying the result images or source images in a different manner, e.g. fragmentary, enlarged, reduced, cropped, compressed, with altered resolution etc.”, where in order to compress a file/image a compression method must be selected beforehand.)
Bruderle discloses 14. The method according to claim 13, wherein the common signal processing device is able to dynamically adapt a structure size and/or encoding of the data to the compression.(This limitation is inherent to compression in that all compression algorithms dynamically encode/reduce the size of the file/image )
Bruderle discloses 15. The method of claim 1, further comprising the steps of: retrieving the stored modified video image data; extracting the further data from the modified video image data; and analyzing the extracted static and measured dynamic data in conjunction with the video image data configured to evaluate one or more of the following assessment of the input variables set for the medical intervention, the selected application modes, the effectiveness of the desired treatment effects, finding optimization possibilities for the set variables, application modes or the medical device or devices, or detection of inadequacies (Bruderle, pg. 4 “The display device 30 has an ad 32 and an image analysis unit 34 trained to code 22
from the result image 26 , which from the storage device 28 is read and the Process data 12 encoded in the code is decoded. The display device 30 further comprises a display control unit 36 which is designed for that in several result images 26 contained source images 16 with the associated, decoded process data over time on the Display 32 display.”, where the image and decoded further data are displayed for future viewing by users for sufficient analysis either immediately or several years later (see pg. 5, paragraph 4)).
Bruderle discloses 16. The method of claim 15, wherein the analyzing the extracted static and measured dynamic data includes detection of one or more of the following inadequacies: electrical limitations for the application, insufficient speed of the treatment effects, occurrence of lateral damage, including carbonization or onset of bleeding, and, if occurrence of lateral damage is detected, further includes determination of causes of the lateral damage or other complications.(As indicated by paragraph 38 of Applicant’s Specification these relationships are performed manually by an operator. The Examiner contends any person2 who watches a video of an operation would inherently do many, if not all these operations, for example determination of bleeding simply requires spotting blood coming out of an organ, insufficient speed could mentally be determined by a layperson as something they think takes too long etc.…)
Claims 17-18 are rejected under similar grounds as claim 1-2, respectively.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GANDHI THIRUGNANAM whose telephone number is (571)270-3261. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GANDHI THIRUGNANAM/ Primary Examiner, Art Unit 2672
1 112 2nd
2 Excluding babies or very small children
Prosecution Insights