DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 9, 11-13, 15 and 22 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Barclay et al. (US 20200121245 A1, Apr. 23, 2020) (hereinafter “Barclay”).
Regarding claim 9: Barclay discloses an image capturing and processing device configured to measure a fluorescence signal in a tissue of a limb, to which a fluorescent agent has been added ([0141], [0271] – the disclosed system can be used with or without agent, which means that it is configured as claimed), and to determine a 3D representation of at least a section of the limb, wherein the tissue to which the fluorescent agent has been added forms part of the limb, the device comprising: an image capturing device ([0077]) comprising: an illumination light source configured to illuminate the tissue with excitation light having a wavelength suitable to generate emitted light by excited emission of the fluorescent agent ([0082], [0133], [0146]), and a fluorescence image sensor configured to capture a fluorescence image by spatially resolved measurement of the emitted light so as to provide the fluorescence image ([0033]-[0034], [0132], [0140]), and a processor comprising hardware ([0075]-[0076]), the processor being configured to: receive data on a topology of the surface of at least the section of the limb and to determine a 3D representation of at least the section of the limb from the captured data ([0080], [0089]), determine a volume of at least the section of the limb from the 3D representation, and output the fluorescence image and the visualization of the 3D representation together with a visualization of the determined volume ([0143], [0160]-[0162], where the wound is at least a section of the limb, figs. 19-23, [0215]); wherein the fluorescence image sensor further outputs the data on the topology of the surface of at least the section of the limb to the processor ([0089] - camera 126 produces the "data on the topology of the surface", [0140] - where camera 126 may also be the "fluorescence sensor").
Regarding claim 11: The device of claim 9, wherein the processing is further configured to: superimpose the fluorescence image and the visualization of the 3D representation of at least the section of the limb so as to provide an overlay image, and output the overlay image as the output of the fluorescence image and the visualization of the 3D representation ([0169], figs. 19-23, [0213]-[0214]).
Regarding claim 13: Barclay discloses the device of claim 9, wherein the image capturing device further comprises: a visible light image sensor configured to capture a visible light image of at least the section of the surface of the limb, wherein the fluorescence image sensor and the visible light image sensor are configured in that a viewing direction and/or a perspective of the fluorescence image and the visible light image are linked via a known relationship, wherein the processor is configured to output the visible light image together with the fluorescence image and the visualization of the 3D representation ([0132] - camera 151 can be used to capture both visible light images and fluorescence images; alternatively, fluorescence images are captured with the 3D camera 126 which has a known relationship to camera 151 - [0140], [0152]-[0154], [0143], [0169], figs. 19-23).
Regarding claim 15: Barclay discloses the device according to claim 13, wherein the fluorescence image sensor and the visible light image sensor are configured in that the viewing direction and the perspective of the fluorescence image and the visible light image are identical, 'wherein the fluorescence image sensor and the visible light image sensor are configured in that the fluorescence image and the visible light image are captured through a same objective lens ([0132] - camera 151 can be used to capture both visible light images and fluorescence images, which means they would use the same optical path including a lens).
Regarding claim 22: Barclay discloses a processing device for measuring a fluorescence signal in a tis- sue of a limb, to which a fluorescent agent has been added, and of determining a 3D representation of at least a section of the limb, wherein the tissue to which the fluorescent agent has been added forms part of the limb ([0141], [0271] – the disclosed system can be used with or without agent, which means that it is configured as claimed), the processing device comprising: a processor comprising hardware ([0075]-[0076]), the processor being configured to: receive a fluorescence image, from a fluorescence image sensor, of the tissue illuminated with excitation light having a wavelength suitable to generate emitted light by excited emission of the fluorescent agent, and by spatially resolved measurement of the emitted light so as to provide the fluorescence image, receive data on a topology of the surface of at least the section of the limb ([0075]-[0077], [0080], [0082], [0089], [0133], [0146]); determine a 3D representation of at least the section of the limb from the received data([0080], [0089]), determine a volume of at least the section of the limb from the 3D representation, and output the fluorescence image and the visualization of the 3D representation together with a visualization of the determined volume ([0143], [0160]-[0162], where the wound is at least a section of the limb, figs. 19-23, [0215]); herein the fluorescence image sensor further outputs the data on the topology of the surface of at least the section of the limb to the processor ([0089] - camera 126 produces the "data on the topology of the surface", [0140] - where camera 126 may also be the "fluorescence sensor").
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 14 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barclay in view of DiMaio et al. (US 2017/0367580 A1, Dec. 28, 2017) (hereinafter “DiMaio”).
Regarding claim 14: Barclay discloses the device according to claim 13, wherein the fluorescence image sensor and the visible light image sensor are further configured to repeat capturing of the fluorescence image and the visible light image to provide a series of fluorescence images and a series of visible light images ([0179]), wherein the processor is configured to: receive the data on the topology of the surface of the limb in at least the section of the limb that is imaged when capturing the series of fluorescence images and the visible light images ([0080], [0089]), and outputting the fluorescence and visible light images with the visualization of the 3D representation (figs. 19-23, [0213]-[0214]). While Barclay suggests combining the image data to map the skin surface based on several images captured from different positions ([0179]), Barclay is silent on any details of this process and does not teach applying a stitching algorithm on the series of visible light images to generate a large visible light image of the limb, the stitching algorithm determining and applying a set of stitching parameters, applying the stitching algorithm on the series of fluorescence im- ages to generate a large fluorescence image, wherein the stitching algorithm applies the set of stitching parameters determined when performing the stitching of the visible light images.
DiMaio, in the same problem field of endeavor, teaches creating a large image mosaic to show larger areas of the body in cases of wide-spread wounds ([0233], [0236], fig. 4), where images may be white light ([0281]) or fluorescence ([0253]), including apply a stitching algorithm on the series of images to generate a large image of the limb, the stitching algorithm determining and applying a set of stitching parameters, wherein the stitching algorithm applies the set of stitching parameters determined when performing the stitching of the images (fig. 4, [0235] - at least the edge detection results and tissue cross-correlation results are "a set of stitching parameters" since they are used to determine how the image sections should be correctly joined, stitched, and pieced together).
It would have been prima facie obvious for one having ordinary skill in the art prior to the effective filing date of the claimed invention to modify the system of Barclay by including the mosaic technique of DiMaio in order to provide a visualization of wounds that are too large to be captured in a single frame.
It is noted that while DiMaio does not explicitly describe performing the stitching algorithm on both visible and fluorescence images, it would have been prima facie obvious to do so in order to provide the complete display of Barclay for a larger-sized wound.
Regarding claim 23: Barclay discloses an image capturing and processing device configured to measure a fluorescence signal in a tissue of a limb, to which a fluorescent agent has been added ([0141], [0271] – the disclosed system can be used with or without agent, which means that it is configured as claimed), and to determine a 3D representation of at least a section of the limb, wherein the tissue to which the fluorescent agent has been added forms part of the limb, the device comprising: an image capturing device ([0077]) comprising: an illumination light source configured to illuminate the tissue with excitation light having a wavelength suitable to generate emitted light by excited emission of the fluorescent agent ([0082], [0133], [0146]), and a fluorescence image sensor configured to capture a fluorescence image by spatially resolved measurement of the emitted light so as to provide the fluorescence image ([0033]-[0034], [0132], [0140]), and a processor comprising hardware ([0075]-[0076]), the processor being configured to: receive data on a topology of the surface of at least the section of the limb and to determine a 3D representation of at least the section of the limb from the captured data ([0080], [0089]), determine a volume of at least the section of the limb from the 3D representation, and output the fluorescence image and the visualization of the 3D representation together with a visualization of the determined volume ([0143], [0160]-[0162], where the wound is at least a section of the limb, figs. 19-23, [0215]); wherein the image capturing device further comprises: a visible light image sensor configured to capture a visible light image of at least the section of the surface of the limb, wherein the fluorescence image sensor and the visible light image sensor are configured in that a viewing direction and/or a perspective of the fluorescence image and the visible light image are linked via a known relationship, wherein the processor is configured to output the visible light image together with the fluorescence image and the visualization of the 3D representation ([0132] - camera 151 can be used to capture both visible light images and fluorescence images; alternatively, fluorescence images are captured with the 3D camera 126 which has a known relationship to camera 151 - [0140], [0152]-[0154], [0143], [0169], figs. 19-23); and wherein the fluorescence image sensor and the visible light image sensor are further configured to repeat capturing of the fluorescence image and the visible light image to provide a series of fluorescence images and a series of visible light images ([0179]), wherein the processor is configured to: receive the data on the topology of the surface of the limb in at least the section of the limb that is imaged when capturing the series of fluorescence images and the visible light images ([0080], [0089]), and outputting the fluorescence and visible light images with the visualization of the 3D representation (figs. 19-23, [0213]-[0214]).
While Barclay suggests combining the image data to map the skin surface based on several images captured from different positions ([0179]), Barclay is silent on any details of this process and does not teach applying a stitching algorithm on the series of visible light images to generate a large visible light image of the limb, the stitching algorithm determining and applying a set of stitching parameters, applying the stitching algorithm on the series of fluorescence im- ages to generate a large fluorescence image, wherein the stitching algorithm applies the set of stitching parameters determined when performing the stitching of the visible light images.
DiMaio, in the same problem field of endeavor, teaches creating a large image mosaic to show larger areas of the body in cases of wide-spread wounds ([0233], [0236], fig. 4), where images may be white light ([0281]) or fluorescence ([0253]), including apply a stitching algorithm on the series of images to generate a large image of the limb, the stitching algorithm determining and applying a set of stitching parameters, wherein the stitching algorithm applies the set of stitching parameters determined when performing the stitching of the images (fig. 4, [0235] - at least the edge detection results and tissue cross-correlation results are "a set of stitching parameters" since they are used to determine how the image sections should be correctly joined, stitched, and pieced together).
It would have been prima facie obvious for one having ordinary skill in the art prior to the effective filing date of the claimed invention to modify the system of Barclay by including the mosaic technique of DiMaio in order to provide a visualization of wounds that are too large to be captured in a single frame.
It is noted that while DiMaio does not explicitly describe performing the stitching algorithm on both visible and fluorescence images, it would have been prima facie obvious to do so in order to provide the complete display of Barclay for a larger-sized wound.
Claim(s) 16 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Barclay in view of Meester (US 2021/0137369 A1, May 13, 2021) (hereinafter “Meester”).
Regarding claim 16: Barclay discloses the device according to claim 13, but discloses that only images with non-overlapping wavelengths may be acquired simultaneously, while images that do have overlapping wavelengths must be taken sequentially ([0236]). Therefore, Barclay does not teach wherein the fluorescence image sensor and the visible light image sensor are configured to capture the fluorescence image and the visible light image simultaneously, in absence of time-switching between a signal of the fluorescence image and a signal of the visible light image. Barclay further teaches that simultaneous image capture is preferred since it avoids the need for registration when there is motion of the device ([0263]).
Meester, in the same problem solving of multi-wavelength imaging, teaches an image capture device comprising multiple image sensors which are configured to capture images having overlapping wavelength ranges simultaneously, in absence of time-switching between a signal of the fluorescence image and a signal of the visible light image ([0011]).
It would have been prima facie obvious for one having ordinary skill in the art prior to the effective filing date of the claimed invention to modify the system of Barclay so that the fluorescence image sensor and visible light image sensor may acquire images simultaneously in absence of time-switching in view of the teachings of Meester, in order to avoid the need for registration when there is motion of the device.
Regarding claim 17: Barclay discloses the device according to claim 9, but does not disclose wherein the image capturing device further comprises: a dichroic prism assembly configured to receive fluorescent light forming the fluorescence image and visible light forming the visible light image through an entrance face, the dichroic prism assembly comprising: a first prism subassembly comprising a first prism, a second prism, a first compensator prism located between the first prism and the second prism, a second dichroic prism subassembly for splitting the visible light in three light components, and a second compensator prism located between the second prism and the second prism subassembly, wherein the first prism and the second prism each have a cross section with at least five corners, each corner having an inside angle of at least 90 degrees, wherein the corners of the first prism and the second prism each have a respective entrance face and a respective exit face, and are each configured so that an incoming beam which enters the entrance face of the respective prism in a direction parallel to a normal of said entrance face is reflected twice inside the respective prism and exits the respective first prism and second prism through its exit face parallel to a normal of said exit face, wherein the normal of the entrance face and the normal of the exit face of the respective first prism and second prism are perpendicular to each other; wherein, when light enters the first prism through the entrance face, the light is partially reflected towards the exit face of the first prism thereby traveling a first path length from the entrance face of the first prism to the exit face of the first prism, and the light partially enters the second prism via the first compensator prism and is partially reflected towards the exit face of the second prism, thereby traveling a second path length from the entrance face of the first prism to the exit face of the second prism, and wherein the first prism is larger than the second prism so that the first and the second path lengths are the same.
Meester, in the same problem solving of multi-wavelength imaging, teaches an image capturing device comprising a dichroic prism assembly configured to receive fluorescent light forming the fluorescence image and visible light forming the visible light image through an entrance face, the dichroic prism assembly comprising: a first prism subassembly comprising a first prism, a second prism, a first compensator prism located between the first prism and the second prism, a second dichroic prism subassembly for splitting the visible light in three light components, and a second compensator prism located between the second prism and the second prism subassembly, wherein the first prism and the second prism each have a cross section with at least five corners, each corner having an inside angle of at least 90 degrees, wherein the corners of the first prism and the second prism each have a respective entrance face and a respective exit face, and are each configured so that an incoming beam which enters the entrance face of the respective prism in a direction parallel to a normal of said entrance face is reflected twice inside the respective prism and exits the respective first prism and second prism through its exit face parallel to a normal of said exit face, wherein the normal of the entrance face and the normal of the exit face of the respective first prism and second prism are perpendicular to each other; wherein, when light enters the first prism through the entrance face, the light is partially reflected towards the exit face of the first prism thereby traveling a first path length from the entrance face of the first prism to the exit face of the first prism, and the light partially enters the second prism via the first compensator prism and is partially reflected towards the exit face of the second prism, thereby traveling a second path length from the entrance face of the first prism to the exit face of the second prism, and wherein the first prism is larger than the second prism so that the first and the second path lengths are the same (fig. 5 and all associated description). Meester further teaches that this arrangement is advantageous because the optical assembly can have sensors mostly on one side, still use an even amount of direction change so all sensors see the same image and no mirror effects need to be compensated for ([0084]) and the prism modules can be easily aligned which makes automatic assembly easy. These modules can be prepared individually and afterwards bonded together with a simple robotic or manual tool ([0090]).
It would have been prima facie obvious for one having ordinary skill in the art prior to the effective filing date of the claimed invention to implement the system of Barclay with a dichroic prism assembly as taught by Meester in order to achieve the advantages of no mirror effect compensation and easy automatic assembly in view of the further teachings of Meester.
Response to Arguments
Rejection of claims 9, 11-17, and 22 under 35 U.S.C. §112(a) is withdrawn in light of the amendments to the claims.
The indicated allowability of former claim 12 (now incorporated into claim 9) and claim 14 is withdrawn in view of the amendments to the independent claims. The claims were previously objected to as being allowable subject matter depending from a rejected claim, which necessarily requires every limitation of the base claims in addition to every limitation of the claims found to contain allowable subject matter. It is the combination of all limitations from the independent and the indicated dependent claims as a whole that make up the allowable subject matter. Changing the scope of the independent claim(s) by removing limitations necessarily changes the scope of the dependent claims, which are no longer indicated as allowable and are now subject to prior art rejections.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Sevick-Muraca et al. (US 2013/0267843 A1, Oct. 10, 2013) – teaches fluorescence imaging for lymphedema evaluation.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLYN A PEHLKE whose telephone number is (571)270-3484. The examiner can normally be reached 9:00am - 5:00pm (Central Time), Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Koharski can be reached at (571) 272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLYN A PEHLKE/ Primary Examiner, Art Unit 3799