DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The term white light image is interpreted based on the definition set forth in paragraph 71 of the published specification: “As described herein, a "white light" image refers to an image of a target area (e.g., tissue) that is illuminated with a substantially white light”
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 9-14, 18-19, are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Tully et al. (US20200315432, hereafter Tully).
Regarding claim 1, Tully discloses a method for visualizing blood flow (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time”), the method comprising:
obtaining a laser speckle contrast imaging (LSCI) image of blood flow (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 108; “In an example, the image sensor of the optical adapter may be used for laser speckle imaging. In such a case, the first set of imaging data may comprise one or more laser speckle patterns”);
obtaining a white light image of a tissue, the white light image capturing an anatomical structure of a subject in a region associated with the LSCI image of the blood flow (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 101; “The second portion of the light signals may comprise reflected light that is generated when the target site is illuminated with a different light (e.g., white light)”) (Tully, Para 108; “second set of imaging data may comprise one or more photographic and/or video images.”);
spatially registering the LSCI image and the white light image with one another (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”);
overlaying the spatially registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”); and
generating display data that continuously depicts the blood flow (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time, […] In an example, a real-time event may comprise visualization of blood perfusion at a tissue site, in which a data set (e.g., one or more light signals) indicative of the blood perfusion is captured by a tool (e.g., an image sensor), and the data is transmitted to a display for visualization to a user. In another example, a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) overlaying the tissue, wherein the display data comprises the spatially registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 155; “The present disclosure provides methods and systems that can be used with commercially available endoscopic devices for displaying a laser speckle contrast image in addition to a standard image of a surgical site. The images may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”).
Regarding claim 2, Tully discloses all of the limitations of claim 1 as discussed above.
Tully further discloses wherein the display data continuously depicts the blood flow overlaying the tissue in real-time during a surgical procedure (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time […] In an example, a real-time event may comprise visualization of blood perfusion at a tissue site, in which a data set (e.g., one or more light signals) indicative of the blood perfusion is captured by a tool (e.g., an image sensor), and the data is transmitted to a display for visualization to a user. In another example, a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) (Tully, Para 69; “The optical adapter of the present disclosure may be usable for a number of medical applications, e.g., general surgery, neurosurgical procedures, orthopedic procedures, and spinal procedures. The optical adapter of the present disclosure may be applicable to a wide variety of endoscopy-based procedures, including, but are not limited to, cholecystectomy (e.g., 1,200,000 procedures per year), hysterectomy (e.g., 575,000 procedures per year), thyroidectomy (e.g., 150,500 procedures per year), and gastrectomy (e.g., 225,000 procedures per year)”) (Tully, Para 94; “The image sensor may be disposable and configured for single use in a medical imaging procedure. Alternatively, the image sensor may be configured to be reusable for a plurality of medical imaging procedures.”).
Regarding claim 3, Tully discloses all of the limitations of claim 1 as discussed above.
Tully further discloses displaying the display data on a user interface (Tully, Para 67; “a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) (Tully, Para 124; “The base module 250 may comprise a processor configured to analyze data obtained by the camera 220, a light source to provide light to the scope assembly 200, a display (e.g., a liquid crystal display (LCD) or light emitting diode (LED) screen) to visualize the data obtained by the camera.”) (Tully, Para 56; “FIGS. 14A, 14B, and 14C illustrate screenshots of an exemplary standard RGB surgical image, laser speckle contrast image, and laser speckle contrast image overlaid on the standard image, in accordance with some embodiments.”) (Tully, Para 155; “The present disclosure provides methods and systems that can be used with commercially available endoscopic devices for displaying a laser speckle contrast image in addition to a standard image of a surgical site. The images may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”).
Regarding claim 4, Tully discloses all of the limitations of claim 1 as discussed above.
Tully further discloses wherein the tissue comprises vasculature (Tully, Para 139; “Examples of the target site within the subject's body can include, but are not limited to, […] superior vena cava, interior vena cava, […] heart, […] arteries, veins, […] healthy tissue, and diseased tissue.”).
Regarding claim 5, Tully discloses all of the limitations of claim 1 as discussed above.
Tully further discloses wherein the tissue is brain tissue (Tully, Para 139; “Examples of the target site within the subject's body can include, but are not limited to, […] brain, […] and diseased tissue.”).
Regarding claim 9, Tully discloses system for blood flow visualization (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time”), the system comprising:
a first light source configured to illuminate blood flow at a target region of a subject; a first camera configured to record a raw laser speckle image of the blood flow ; and a computing device comprising a processor and a memory, the memory having instructions stored thereon that, when executed by the processor, cause the computing device to (Tully, Para 38-39; “Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods above or elsewhere herein. Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.”): obtain, via the first camera, the raw laser speckle image of the blood flow; derive a laser speckle contrast imaging (LSCI) image from the raw laser speckle image (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 108; “In an example, the image sensor of the optical adapter may be used for laser speckle imaging. In such a case, the first set of imaging data may comprise one or more laser speckle patterns”);
obtain a white light image of tissue at the target region of the subject, the white light image capturing an anatomical structure (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 108; “second set of imaging data may comprise one or more photographic and/or video images.”) (Tully, Para 101; “The second portion of the light signals may comprise reflected light that is generated when the target site is illuminated with a different light (e.g., white light)”);
spatially register the LSCI image and the white light image with one another (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”);
overlay the spatially registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”); and
generate display data that continuously depicts the blood flow (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time, […] In an example, a real-time event may comprise visualization of blood perfusion at a tissue site, in which a data set (e.g., one or more light signals) indicative of the blood perfusion is captured by a tool (e.g., an image sensor), and the data is transmitted to a display for visualization to a user. In another example, a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) overlaying the tissue, wherein the display data comprises the spatially registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 155; “The present disclosure provides methods and systems that can be used with commercially available endoscopic devices for displaying a laser speckle contrast image in addition to a standard image of a surgical site. The images may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”).
Regarding claim 10, Tully discloses all of the limitations of claim 9 as discussed above.
Tully further discloses wherein the system is a surgical microscope, an endoscope, an exoscope, a robotic surgery platform, a stand-alone imaging system, or system dedicated for blood flow imaging (Tully, Para 137; “In some cases, the optical adapter of the present disclosure may collect surgical data that is multiple orders of magnitude denser or more detailed compared to data collected from an existing imaging system (e.g., endoscope or dye-based imaging systems).”) (Tully, Para 71; “The optical adapter (or at least a portion of the optical adapter) may be reused, and may be interchangeable with different scope assemblies. In some cases, the optical adapter may allow a scope from a first scope assembly to be operatively coupled to a camera of a different scope assembly, to thereby further diversifying imaging modalities of existing scope assemblies.”).
Regarding claim 11, Tully discloses all of the limitations of claim 9 as discussed above.
Tully further discloses wherein the display data continuously depicts the blood flow overlaying the tissue in real-time during a surgical procedure (Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time […] In an example, a real-time event may comprise visualization of blood perfusion at a tissue site, in which a data set (e.g., one or more light signals) indicative of the blood perfusion is captured by a tool (e.g., an image sensor), and the data is transmitted to a display for visualization to a user. In another example, a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) (Tully, Para 69; “The optical adapter of the present disclosure may be usable for a number of medical applications, e.g., general surgery, neurosurgical procedures, orthopedic procedures, and spinal procedures. The optical adapter of the present disclosure may be applicable to a wide variety of endoscopy-based procedures, including, but are not limited to, cholecystectomy (e.g., 1,200,000 procedures per year), hysterectomy (e.g., 575,000 procedures per year), thyroidectomy (e.g., 150,500 procedures per year), and gastrectomy (e.g., 225,000 procedures per year)”) (Tully, Para 94; “The image sensor may be disposable and configured for single use in a medical imaging procedure. Alternatively, the image sensor may be configured to be reusable for a plurality of medical imaging procedures.”).
Regarding claim 12, Tully discloses all of the limitations of claim 9 as discussed above.
Tully further discloses wherein the instructions further cause the computing device to present the display data on a user interface (Tully, Para 67; “a real-time event may comprise combining two different data sets that are indicative of different features of the tissue site for a simultaneous visualization at the display.”) (Tully, Para 124; “The base module 250 may comprise a processor configured to analyze data obtained by the camera 220, a light source to provide light to the scope assembly 200, a display (e.g., a liquid crystal display (LCD) or light emitting diode (LED) screen) to visualize the data obtained by the camera.”) (Tully, Para 56; “FIGS. 14A, 14B, and 14C illustrate screenshots of an exemplary standard RGB surgical image, laser speckle contrast image, and laser speckle contrast image overlaid on the standard image, in accordance with some embodiments.”) (Tully, Para 155; “The present disclosure provides methods and systems that can be used with commercially available endoscopic devices for displaying a laser speckle contrast image in addition to a standard image of a surgical site. The images may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”).
Regarding claim 13, Tully discloses all of the limitations of claim 9 as discussed above.
Tully further discloses wherein the tissue comprises vasculature (Tully, Para 139; “Examples of the target site within the subject's body can include, but are not limited to, […] superior vena cava, interior vena cava, […] heart, […] arteries, veins, […] healthy tissue, and diseased tissue.”).
Regarding claim 14, Tully discloses all of the limitations of claim 9 as discussed above.
Tully further discloses wherein the tissue is brain tissue (Tully, Para 139; “Examples of the target site within the subject's body can include, but are not limited to, […] brain, […] and diseased tissue.”).
Regarding claim 18, Tully discloses method for visualizing blood flow(Tully, Para 67; “The optical adapter of the present disclosure may allow visualization of perfusion (e.g., blood perfusion) at a tissue site of interest substantially in real-time”), the method comprising continuously:
capturing, using a first image capture device, a laser speckle contrast imaging (LSCI) image of blood flow in a subject (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 108; “In an example, the image sensor of the optical adapter may be used for laser speckle imaging. In such a case, the first set of imaging data may comprise one or more laser speckle patterns”);
capturing, using a second image capture device, a white light image of a tissue, the white light image capturing an anatomical structure (Tully, Para 8; “the image sensor is configured to generate a first set of imaging data from the first portion of the light signals, and the camera is configured to generate a second set of imaging data from the second portion of the light signals. In some embodiments, the first set of imaging data comprises laser speckle patterns, and the second set of imaging data comprises photographic or video images.”) (Tully, Para 108; “second set of imaging data may comprise one or more photographic and/or video images.”) (Tully, Para 101; “The second portion of the light signals may comprise reflected light that is generated when the target site is illuminated with a different light (e.g., white light)”);
co-registering the LSCI image and the white light image (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”);
overlaying the co-registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”); and
displaying, via a user interface, the overlayed and co-registered LSCI and white light images (Tully, Para 117; “Such alignment of the two imaging units may be useful when creating an overlay of the first set and the second set of imaging data, e.g., when generating an overlay of blood flow and perfusion (e.g., from the image sensor) on top of the standard white light surgical view (e.g., from the camera).”) (Tully, Para 155; “The present disclosure provides methods and systems that can be used with commercially available endoscopic devices for displaying a laser speckle contrast image in addition to a standard image of a surgical site. The images may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”) (Tully, Para 161; “The laser speckle contrast images and the standard RGB images of the surgical site may be displayed individually or together. For instance, the laser speckle contrast image may be overlaid on the standard image of the surgical site.”).
Regarding claim 19, Tully discloses all of the limitations of claim 18 as discussed above.
Tully further discloses wherein the tissue comprises vasculature or brain tissue (Tully, Para 139; “Examples of the target site within the subject's body can include, but are not limited to, […] superior vena cava, interior vena cava, […] heart, […] brain, […] arteries, veins, […] healthy tissue, and diseased tissue.”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-7, 15-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tully and Andersson et al. (US20200279414, hereafter Andersson).
Regarding claim 6, Tully discloses all of the limitations of claim 1 as discussed above.
Tully discloses registering the LSCI image and the white light image with one another using a spatial transformation (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”).
Tully does not clearly and explicitly disclose creating a lookup table based on the spatial transformation used to register the LSCI and white light images.
In an analogous transformation of a medical image field of endeavor Andersson discloses comprises creating a lookup table based on a spatial transformation used (Andersson, Para 99; “A common way of converting the CBCT image is to create a value transformation, for example in the form of a lookup table (LUT) mapping values in the CBCT image to values in corresponding voxels of the CT image. Such a value transformation may be obtained in different ways”) (Andersson, Para 29; “is typically performed by means of a transformation representation such as a function, a lookup table (LUT) or a diagram, said transform representation mapping a value in the first CT image to a value in the second CT image or vice versa, and applying that transformation representation to the image that is to be converted”) (Andersson, Para 100; “FIG. 4 discloses a method of obtaining a representation that may be used to perform a transformation from CBCT values to CT values, or vice versa. The representation may be presented in the form of a function, a lookup table, a diagram or in any other suitable way.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully to include creating a lookup table based on the spatial transformation used to register the LSCI and white light images as taught by Andersson in order to allow for reduced computational load, faster data retrieval, and greater efficiency.
Regarding claim 7, Tully as modified by Andersson discloses all of the limitations of claim 6 as discussed above.
Tully as modified by Andersson above discloses wherein overlaying the spatially registered LSCI and white light images comprises mapping respective pixels from the LSCI image to respective pixels from the white light image using the lookup table (Tully, Para 117; “processor may perform digital image processing on one or both of the first set and the second set of imaging data (e.g., affine transformation of one or more pixels of the first set and the second set of imaging data), such that the perspectives of the image sensor and the camera are aligned (or lined up) and spatially correspond to each other”).
Tully as modified by Andersson above is interpreted as disclosing these limitations in the claim because Tully transforms pixels and Andersson modifies Tully to use a lookup table to do so.
Regarding claim 15, Tully discloses all of the limitations of claim 9 as discussed above.
Tully discloses spatially registering the LSCI image and the white light image with one another using a spatial transformation (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”).
Tully does not clearly and explicitly disclose creating a lookup table based on the spatial transformation used to register the LSCI and white light images.
In an analogous transformation of a medical image field of endeavor Andersson discloses comprises creating a lookup table based on a spatial transformation used (Andersson, Para 99; “A common way of converting the CBCT image is to create a value transformation, for example in the form of a lookup table (LUT) mapping values in the CBCT image to values in corresponding voxels of the CT image. Such a value transformation may be obtained in different ways”) (Andersson, Para 29; “is typically performed by means of a transformation representation such as a function, a lookup table (LUT) or a diagram, said transform representation mapping a value in the first CT image to a value in the second CT image or vice versa, and applying that transformation representation to the image that is to be converted”) (Andersson, Para 100; “FIG. 4 discloses a method of obtaining a representation that may be used to perform a transformation from CBCT values to CT values, or vice versa. The representation may be presented in the form of a function, a lookup table, a diagram or in any other suitable way.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully to include creating a lookup table based on the spatial transformation used to register the LSCI and white light images as taught by Andersson in order to allow for reduced computational load, faster data retrieval, and greater efficiency.
Regarding claim 16, Tully as modified by Andersson discloses all of the limitations of claim 15 as discussed above.
Tully as modified by Andersson above discloses wherein overlaying the spatially registered LSCI and white light images comprises mapping respective pixels from the LSCI image to respective pixels from the white light image using the lookup table (Tully, Para 117; “processor may perform digital image processing on one or both of the first set and the second set of imaging data (e.g., affine transformation of one or more pixels of the first set and the second set of imaging data), such that the perspectives of the image sensor and the camera are aligned (or lined up) and spatially correspond to each other”).
Tully as modified by Andersson above is interpreted as disclosing these limitations in the claim because Tully transforms pixels and Andersson modifies Tully to use a lookup table to do so.
Regarding claim 20, Tully discloses all of the limitations of claim 18 as discussed above.
Tully discloses co-registering the LSCI image and the white light image with one another using a spatial transformation (Tully, Para 117; “the processor may be configured to perform image registration. The processor may be configured to find one or more matching features in the first set and the second set of imaging data, then calculate a transformation of one or both of the first set and the second set of imaging data for their alignment”).
Tully does not clearly and explicitly disclose creating a lookup table based on the spatial transformation used to register the LSCI and white light images.
In an analogous transformation of a medical image field of endeavor Andersson discloses comprises creating a lookup table based on a spatial transformation used (Andersson, Para 99; “A common way of converting the CBCT image is to create a value transformation, for example in the form of a lookup table (LUT) mapping values in the CBCT image to values in corresponding voxels of the CT image. Such a value transformation may be obtained in different ways”) (Andersson, Para 29; “is typically performed by means of a transformation representation such as a function, a lookup table (LUT) or a diagram, said transform representation mapping a value in the first CT image to a value in the second CT image or vice versa, and applying that transformation representation to the image that is to be converted”) (Andersson, Para 100; “FIG. 4 discloses a method of obtaining a representation that may be used to perform a transformation from CBCT values to CT values, or vice versa. The representation may be presented in the form of a function, a lookup table, a diagram or in any other suitable way.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully to include creating a lookup table based on the spatial transformation used to register the LSCI and white light images as taught by Andersson in order to allow for reduced computational load, faster data retrieval, and greater efficiency.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Tully and Andersson as applied to claims 7 and 16 above, and further in view of Wei et al. (US20120194783, hereafter Wei), 김정배 (KR20070016664, hereafter ‘664), and Silberman et al. (US20190282208, hereafter Silberman).
Regarding claims 8 and 17, Tully as modified by Andersson discloses all of the limitations of claims 7 and 16 as discussed above.
Tully does not clearly and explicitly disclose wherein overlaying the spatially registered LSCI and white light images further comprises contrast stretching the LSCI image, mapping the LSCI image to an n-bit color map, and performing a weighted sum with the white light image.
In an analogous medical imaging device field of endeavor Wei discloses contrast stretching an image (Wei, Para 30; “One of the simplest examples is contrast stretching; where a transformation function stretches a portion of the image histogram for amplitudes that contain desired information are placed across the whole amplitude range”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully wherein overlaying the spatially registered LSCI and white light images further comprises contrast stretching the LSCI image in order to can accentuate features of interest and facilitate diagnosis of data as taught by (Wei, Para 30).
In an analogous imaging device field of endeavor ‘664 discloses mapping an image to an n-bit color map (‘664, Pg 4; “As described above, according to the present invention, by determining whether each pixel corresponds to a skin color in an input color image by using an N-bit RGB map generated in advance from skin color models defined in various color spaces”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully to include mapping the LSCI image to an n-bit color map in order to more quickly and accurately perform image analysis as taught by ‘664 (‘664, Pg 4).
In an analogous medical imaging device field of endeavor Silberman discloses performing a weighted sum with an image (Silberman, Para 11; “generating the composite of the first ultrasound image and the transformed version of the first ultrasound image includes generating a weighted sum of the first ultrasound image”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tully to include performing a weighted sum with the white light image as taught by Silberman in order to improve image quality and SNR.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to John Li whose telephone number is (313)446-4916. The examiner can normally be reached Monday to Thursday; 5:30 AM to 3:30 PM Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN D LI/Primary Examiner, Art Unit 3798