DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-6, 8-9, and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over US 20130245456 A1 by Ferguson et al. (hereafter Ferguson, previously of record) in view of US 20180020932 A1 by Chen et al. (hereafter Chen, previously of record) and further in view of The Fast Fourier Transform in Hardware: A Tutorial Based on an FPGA Implementation by Slade et al. (hereafter Slade, previously of record) and further in view of WO 2019104476 A1 by Qing et al. (hereafter Qing, newly of record, machine translation attached).
Regarding claims 1 and 9, Ferguson teaches: 1. A surgical visualization system to analyze at least a portion of a surgical field, the system comprising:
a laser-light illumination source configured to illuminate the at least the portion of the surgical field with laser-light (see Ferguson’s Fig. 1 noting source 120 includes laser 123 configured to illuminate ROI 140 which, as per [0119] can be a vessel or organ of a patient);
a light sensor configured to receive reflected laser-light (see Ferguson’s Fig. 1 noting camera 130); …
a first display, positioned within at least the portion of the surgical field (just as with the applicant’s specification Ferguson teaches that the display can be in the room with the patient/at the site of measurement, thus in a portion of the surgical field, as per Figs. 21-24 or 49 or 53 among others where computer 2110 or console (unlabeled) contains a clearly visible display; while these figures show the setup without a patient present, see also [0114], [0197], [0204], [0206], [0210], or especially [0211] which describe that this is portable and intended for use in the surgical field during surgery and the last of which describes images taken by the system during a surgical procedure per se), configured to display an output that represents the moving particle information in the at least the portion of the surgical field (see Ferguson’s Fig. 2B noting display 345 and further noting that as claim 1 has been amended to clarify that the processor forms specific outputs, see also Figs 61-62 which, as described in [0223] are average (i.e. aggregated) flow maps and see also Figs. 58-59 which, as described in [0221[ are instantaneous (i.e. present) flow maps)); and
a processor (see Ferguson’s [0108]-[0113] which describe that the acts performed by his invention are implemented on one or more processors or other programmable data processing apparatuses), configured to:
determine to operate in default mode of operation; based on the determination to operate in the default mode of operation, apply a first input and a first transform to generate a first [flow/perfusion data] …, wherein the first [flow/perfusion data] represents a present state of the moving particle information the at least a portion of the surgical field, and wherein the present state of the moving particle information is the output displayed; receive a user input via an input device; determine to operate in an alternative mode of operation based on the received user input based on the determination to operate in the alternative mode of operation, change at least one of the first input to a second input or the first transform to a second transform to generate a second [flow/perfusion data], wherein the second [flow/perfusion data] represents an aggregated state of the moving particle information, and wherein the aggregated state of the moving particle information is the output displayed (regarding these together, see Ferguson’s [0129]-[0131] noting the use of a touch screen for receiving user input and/or that a keyboard or the like can be used to take in user input. Then note that Ferguson teaches that the processor can create both instantaneous and time averaged maps, e.g. see [0031]-[0032] and note that the method includes generating any of instantaneous flow maps, instantaneous perfusion maps, average flow maps, and average perfusion maps for the region of interest. As such the processor is configured to operate in multiple modes where both the first and second modes represent the state of moving particles in the blood/surgical field and where these are specifically present states and aggregate/average states. Furthermore the use of the user input to change modes is also covered in many sections of Ferguson such as [0220] and Fig. 57 where the user can use and EKG display to select cardiac cycle data and thus determine which input data will be used by the processor such that it is clear that the user can select both the input data and the first/second mode) ….
In the forgoing the examiner replaced “doppler” with [flow/perfusion data] because Ferguson teaches using LSI not LDI. In this instance the examiner notes that LSI and LDI are both very well known for their ability to gather flow and perfusion data and that both are commonly used. In this instance the examiner has declined to rely solely on MPEP 2144.03 because this is in the independent claim and because references are readily available as the equivalency and substitutability of LDI and LSI for providing blood flow and perfusion information is taught in myriad arts such as for instance Chen. More specifically, Chen is in the related field of blood flow imaging (See Chen’s Abstract) and teaches both that LDI and LSI (among other method) can be used to gather blood flow and perfusion information (see e.g. [0002]. [0014], [0060], which use these as equivalently substitutable means to gather flow and perfusion data, or see [0064] which acknowledges that they can be used together to gather flow and perfusion data. From there they are no longer discussed in terms of being equivalents but Chen continues to use them interchangeable as the input for his other elements such as the display and mapping of [0095] or [0098] etc.) and goes on the teach the specifics of how to implement both LSI and LDI such that there can be no question that one of ordinary skill in the art, well before the date of invention, would have been well appraised of exactly how to go about switching the processing from LSI to LDI (see Chen’s [0162] noting that the generic laparoscopic and benchtop apparatuses can use either LSI or LDI and that in order to change between them one needs to use different data processing algorithms, specifically equations 1 for LSI or equations 2 for LDI, which in light of [0077]-[0081] clarifies that there are no realistic barriers to implementation as the modification to the processing is known and simple).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art prior to the date of invention to modify Fergusons’ LSI processing to either be (see MPEP 2144.06(II)) or further include (see MPEP 2144.06(I)) LDI, as taught by Chen, because the equivalence and suitability of both methods for providing flood flow and perfusion information is known in the art as recognized by Chen.
Additionally the examiner notes that Ferguson teaches that the processes of his invention can be implemented on multiple structures which can include data processing systems, computers, hardware, circuits, local and/or remote computers, plural computers, general-purpose computers, special purpose computers, or other programmable data processing apparatus (see Ferguson’s [0108]-[0113], noting that ‘circuits’ is a genus covers FGPAs and that ‘other programmable data processing apparatuses’ appears to be at the very least a synonym for FGPA as in order to be a programmable processing apparatus that is not a computer it appears one would have to be discussing FGPAs per se). Likewise the examiner also notes that Ferguson teaches that his first mode is “instantaneous” as iterated above and clarifies in many places that this is to be done in real-time (see e.g. Ferguson’s [0210], [0219], or [0230]) while also using the more complex spatio-temporal contrast calculation (e.g. [0232], necessitating FT of large data sets and other time consuming/computationally intensive calculations that must be performed for each pixel in the sensor). Likewise it is already taught in the foregoing that this transforms optical data into flow data, but for compact prosecution one could see Ferguson’s very Abstract to determine that this is LSCI. However Ferguson does not explicitly state that his invention includes [claim 1, omitted as indicated by ellipsis above] “a field programmable gate array configured to transform reflected laser-light information to moving particle information in the at least the portion of the surgical field” or that this generates the first doppler shift or a [from claim 9] “wherein the processor is further configured to enable communication of data from the light sensor to one or more logic elements of the field programmable gate array” and therefore Ferguson does not explicitly teach all claimed limitations.
However Slade, in the related field of digital signal processing demonstrates that when solving similar problems (see Slade’s Abstract and note that “In digital signal processing, the FFT is one of the most fundamental and useful building blocks available” and “FFT in hardware (i.e. in digital logic, field programmable gate arrays, etc.) is useful for high-speed real time processing” establishing that this art is in the DSP field and also that it relates to real time processing, so as to be in a related field and solving a related problem respectively) one could employ FGPAs to perform the necessary computations (see the Abstract for a summary and/or see III. The FFT in Hardware from pages 8-19 for a full disclosure of how to enact such processing) and that this uses parallel processing and also Slade goes on to teach that this is advantageous (note that Slade’s Abstract iterates that “The hardware FFT performs many of its processing tasks in parallel, hence can achieve order-of magnitude throughput improvements over software FFTs executed in DSP microprocessors” As such one of ordinary skill in the art would be motivated to both use a hardware implementation for FFTs or other similar computations and also to connect each of the output of the image sensor’s elements to the logic/programmable elements of the FPGA in the manner recited in claim 9 as this allows for the parallel processing that is so much faster than the digital/serial processing).
Therefore it would have been obvious to one of ordinary skill in the art prior to the date of invention to connect use an FGPA in particular, instead of a different other computer element from among Ferguson’s stated options, connected so as to allow parallel computations and thereby have the plurality of measurement detectors be each coupled to a respective programmable element of the field programmable gate array, as taught by Slade, in order to perform the computations to transform the optical data into information indicative of the movement of particles [which are already taught by Ferguson above] more quickly than they could be implemented in software.
Additionally the examiner omitted the new limitation ”based on the determination to operate in the alternative mode of operation and in response to a user indication at the first display, send visualization control data from the first display to a second display, wherein the visualization control data is configured to control display content of the second display and wherein the second display is positioned outside of at least the portion of the surgical field” because while Ferguson provides some additional relevant teachings (Ferguson states that the first computer/first display can be connected to a computer network and has I/O ports to communicate therewith and specifically to transfer information to other computers on the network, see Ferguson’s [0129] which states as much directly) Ferguson alone does not teach that the computer is configured to switch to or activate other secondary displays and thus would fail to fully teach the claimed limitation.
However, Qing in the related field of image processing and display of medical/physiological images on a network (see Qing’s Abstract) provides additional teachings relevant to the problem of transferring data between computers on a network including teaching how and explaining why one would want to switch display screens (from [0002]-[0007] Qing sets forth that there is a need in the art to have displays near a patient but that it is also useful to display this information at other locations, often on larger screens removed from the patient, Qing then sets forth in [0007] that this is convenient and avoids the issues previously mentioned and then describes how to set up and program a computer to detect switching instructions from a user in [0079]. Likewise, and relevant to certain dependent claims this control/switching is explicitly designed to be conveniently reversable, e.g. at [0093] or [0109]).
Therefore it would have been obvious to one of ordinary skill in the art prior to the date of invention to further modify Ferguson with the teachings of Qing as this would advantageously allow the user to select whichever monitor on the network would be most opportune to use as described above.
Regarding claim 3, Ferguson further teaches: 3. The system of claim 1, wherein the alternate mode of operation differs from the default mode of operation in any of a duration of time or a difference in laser-light frequency (see the rejection of claim 1, noting that the average flow/perfusion maps are generated using a longer duration of data acquisition by definition; however and for compact prosecution purposes see Ferguson’s [0027] and [0029] which cover the average being over multiple cardiac cycles and multiple timepoints before/after a treatment respectively and where instantaneous means exactly its ordinary meaning, e.g. as given in [0216], wherein a gated acquisition over a very short time period relating to e.g. diastole only can be gathered).
Regarding claims 4 Ferguson further teaches: The system of claim 1, wherein the first input is a first light pattern and the second input is a second light pattern (the examiner notes that many things can be a pattern, one of which is the illumination in time, thus teaching a timing pattern not tied to the cardiac cycle could be a first pattern, see [0029], and a timing pattern tied to the cardiac cycle could be a second pattern, see [0027], or vice versa. For compact prosecution purposes the examiner notes that these patterns can be diverse and can repeat according to specific circumstances, e.g. see Figs. 57 and 60 for multiple timing patterns for different cardiac information).
Regarding claim 5, Ferguson IVO Chen and IVO Slade teaches the basic invention of claim 1, but none of the references use software service tiers as the bar for accessing the alternative mode of operation such that they fail to teach: “5. The system of claim 1, wherein the user input comprises a purchased service tier associated with any of a user or instrument, and wherein the purchased service tier is determined to be compatible with the alternative mode of operation.”
However, the examiner notes that tiered access is a basic concept of the software as a service paradigms which is old, well-known, and nearly ubiquitous in all computer/programming fields and it is well-known how and why to implement such tiered access. For instance it provides users with the ability to pay for only services they need and provides companies with a larger range of customers without the need to develop or maintain different products. See MPEP 2144.03.
Therefore it would have been prima facie obvious to one of ordinary skill in the art prior to the date of invention to implement Ferguson’s program, which has a computer capable of multiple modes of operation, as multiple tiers in order to advantageously be able to offer multiple products without having to separately develop and maintain multiple programs.
Regarding claim 6 and 21-22, Ferguson further teaches: 6. The system of claim 1, wherein the second input and the first transform are used to detect the second doppler shift. 21. The surgical system of claim 1, wherein the first input and the second transform are used to detect the second doppler shift. 22. The surgical system of claim 1, wherein the second input and the second transform are used to detect the second doppler shift (regarding these together, the examiner noes that Ferguson teaches >2 transforms the result in the second (averages/aggregated) shift and >2 possible inputs for both transforms so as to teach the entire range covered as per the rejection of claim 1, again noting that: Ferguson [0031]-[0032] teaches generating both average flow maps and average perfusion maps for the region of interest so as to have multiple transforms that result in the second shift. Likewise Ferguson further teaches that these can use additional transforms (e.g. LSSCI and LSCTI transforms are additionally used on the average flow measures in [0223]). Likewise the processor is configured to use multiple inputs including sequences not tied to the cardiac cycle, see [0029], sequencies tied to the cardiac cycle, see [0027], and various particular or iterations of portions of the cardiac cycle, see [0222]-[0223], etc. so as to have a plurality of inputs that can each be applied to the plurality of transforms to yield the second shift).
Regarding claim 8, Ferguson further teaches: 8. The system of claim 1, wherein the display is configured to display the present state of the moving particles information and the aggregated state of the moving participle information as an overlay on an image that comprises the at least a portion of the surgical field (the examiner notes that this is drafted as function of the display not a processing step (noting that as claim 1 has been amended to clarify that the processor forms these specific outputs, see also Figs 61-62 which, as described in [0223] are average (i.e. aggregated) flow maps and see also Figs. 58-59 which, as described in [0221[ are instantaneous (i.e. present) flow maps).
Regarding claims 24-26 together, Qing further teaches: 24. The surgical system of claim 1, wherein: the user indication indicates a request of changing the display content on the second display, and the visualization control data is configured to change the control display content on the second display based on the user indication. 25. The surgical system of claim 1, wherein: the user indication indicates projecting content associated with the first display onto the second display, and the visualization control data is configured to project the content associated with the first display onto the second display. 26. The surgical system of claim 1, wherein: the user indication indicates removing content associated with the first display from the second display, and the visualization control data is configured to remove the content associated with the first display from the second display (in the same modification made above in regards to claim 1, the examiner noted that Qing’s teachings allow one to request a change of the display content of a second display from a fist display using user indications, see Qing’s [0079] which teaches as much and therefore covers claim 24, as well as teaching that the user can cause the images from the first display to be transferred to the second display or back from the second display to the first display teaching claims 25-26 respectively as iterated in Qing’s in [0093] or [0109]).
Claim(s) 2 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Ferguson IVO Chen IVO Slade as applied to claim 1 above, and further in view of US 20190201104 A1 by Shelton et al. (hereafter Shelton, newly of record).
Regarding claims 2 and 23, Ferguson IVO Chen and IVO Slade teaches the basic invention as given above in regards to claim 1; and Ferguson further teaches that the invention is capable of using multiple wavelengths of light (see e.g. Ferguson’s [0017] noting: “The laser may have a … variable wavelength of from about 600 nm to about 1100 nm.”) however Ferguson does not explicitly teach controlling the laser to output plural wavelengths or processing using inputs of two wavelengths or that these wavelengths will penetrate to different depths (though that is inherent) and thus could be used in depth calculations therefore fails to fully teach: “The system of claim 1, wherein the first input is a first wavelength and the second input is a second wavelength” or “23. The surgical system of claim 1, wherein the processor is further configured to calculate a depth of tissue feature based the first doppler shift and the second doppler shift.”
However Shelton when addressing shortcomings in the same or eminently related field of optical blood flow imaging (see e.g. [0486]) teaches that one can use different wavelengths of light to penetrate to different specific tissue depths (see [0486] noting in particular that: “It is recognized that the tissue penetration depth of light is dependent on the wavelength of the light used. Thus, the wavelength of the laser source light may be chosen to detect particle motion (such a blood cells) at a specific range of tissue depth.” And that Shelton goes on to describe that various wavelengths that fall within the same range already employed by the variable laser of the base art can be used to detect flow at specific depths including giving numerical depth calculations). Shelton goes on to teach advantages of this setup that are in addition to those inherent advantages that come with having more information (see [0491]-[0492] noting that old approaches lead to difficulties in surgery because the surgeon cannot accurately visualize the depth of the vessel, which is solved by interrogating with plural wavelengths of light to reveal the depth of the vessels. Additionally and regardless of the teachings of the art the examiner notes that one of any skill level in the art would immediately recognize the inherent advantage of having a third dimension to any data set. Likewise there are specific obvious surgical applications to having depth data of blood flow or blood perfusion data. For example, without proper depth information a surgeon attempting to avoid a vessel but still needing to access an area past the vessel may end up cutting a blood vessel/leaving an area inadequately perfused which can be directly life threatening due to blood loss in the former situation and due to necrosis in the latter situation such that known how deep a vessel is within the patient is a fundamental and profound advantage that would be well understood by doctors to be potentially lifesaving since it allows the doctor to avoid surgical complications while still making necessary cuts).
Therefore it would have been obvious to one of ordinary skill in the art to operate Fergusons’ laser at multiple wavelengths and to use these to calculate the depth of tissue features such as blood vessels as taught by Shelton in order to advantageously allow for better surgical planning or simply to better understand he patient physiology or avoid surgical complications.
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Ferguson IVO Chen IVO Slade as applied to claim 1 above, and further in view of US 7496395 B2 by Serov et al. (hereafter Serov, previously of record).
Regarding claim 7, Ferguson IVO Chen and IVO Slade teaches the basic invention as given above in regards to claim 1 however, Ferguson uses a generic “camera” instead of the species of a CMOS sensor Ferguson measures the velocity but don’t not specifically state that this is based on the Doppler effect and therefore fails to explicitly teach the limitation: “The system of claim 1, wherein the light sensor comprises a complementary metaloxidesemiconductor (CMOS) imaging sensor array configured to detect the first doppler shift and the second doppler shift”.
However Serov in the same or eminently related field of laser flow imaging (see Serov’s Abstract) teaches that one can use a CMOS sensor array as the camera and that it is advantageous to do so (see Serov’s col. 4 lns. 42-63 which iterate that “CMOS (Complementary Metal Oxide Semiconductor) image sensors progressively developed over the last few years. CMOS image sensor devices are in some way very similar to CCDs. Both technologies are based on photosensitive diodes or gates in silicon. In both solid state devices photons are converted into an electrical charge. The main difference between CMOS and CCD image sensors is basically, that a CMOS image sensor is a 2-D matrix of photodiodes, which can be addressed randomly at a high sampling rate (as described by B. Dierickx et al., in "Random addressable active pixel image sensors", Proc. SPIE 2950, 2-7 (1996)), while in CCD sensors the sampling rate of one pixel is limited to the frame rate. Therefore, CMOS image sensors are able to make both a photographic image of the object under interest and detect rapid intensity changes at each point of the object. Another essential difference is the integration time inherent to CCD but not to all CMOS image sensors. In CMOS image sensors, the photo-current is continuously converted into output voltage as opposed to CCD in which the current is accumulating during a certain period of time that limits the dynamical range of the system, as encountered by Fujii et al.” Furthermore, the examiner notes that Serov explicitly lays out that CMOS sensors of this sort are explicitly capable of measuring the doppler shift , e.g. see Serov’s Title or Abstract or Claim 1 (or claim 3 for this being per CMOS element), etc.).
Therefore it would have been obvious to one of ordinary skill in the art prior to the date of invention to improve the invention of Ferguson by substituting the camera for a CMOS sensor array as taught by Servo in order to advantageously increase the image sampling rate and allow for a higher dynamic range.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and its dependents have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In more detail, the same base references of Ferguson, Chen, and Slade are still used by the examiner, but as correctly pointed out by the applicant in the arguments, these references do not teach the newly claimed limitation of claim 1 nor the limitations found in claims 1 and 24-26. As such a new ground of rejection is provided which addresses this new subject matter but which does not rely on Ferguson, Chen, or Slade to teach the limitation and instead relies on the new reference Qing.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael S Kellogg whose telephone number is (571)270-7278. The examiner can normally be reached M-F 9am-1pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui Pho can be reached at (571)272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL S KELLOGG/Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798