Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Cited Prior Art
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. (See MPEP 2141.02(VI)) Therefore, Applicant is advised to review all relevant portions of the cited prior art if traversing a rejection based on the cited prior art.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7, 8, 11, 85, 88, 91-93, 95 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jones US20200197295, and further in view of Darty US20150308896, Freeman US10646109, Tearney US20180160965, and Ohno US20180239944.
Jones discloses for claim 1, “A capsule hyperspectral system, comprising:
an imaging capsule (ingestible device 100; 0351 describes ingestible device 100 may have a housing 102 shaped similar to a pill or capsule; 0512 describes the corresponding detectors for spectroscopy and hyperspectral imaging), comprising:
an illumination system (illuminators 124; 0355 describes multiple sensing sub-units, each with a separate illuminator and detector) having a plurality of light emitters configured for emitting a plurality of different lighting illuminations from the imaging capsule (0349 describes multiple different lights; 0357 ); and
a hyperspectral imaging system having at least one imaging sensor (0512: ingestible devices can be equipped with sources for generating reflected light, including light in the Ultraviolet, Visible, Near-infrared and/or Mid-infrared spectrum, and the corresponding detectors for spectroscopy and hyperspectral imaging), and
a hyperspectral processing system having at least one processor (0458]- The techniques described above can be implemented using software for execution on a computer. For instance, the software forms procedures in one or more computer programs that execute on one or more programmed or programmable computer systems... each including at least one processor)”.
Jones does not disclose:
wherein the illumination system and hyperspectral imaging system are cooperatively configured to illuminate a target with a sequence of different lighting illuminations and image the target during each of the different lighting illuminations in the sequence
wherein the hyperspectral processing system is operably coupled with the hyperspectral imaging system and configured to receive images of the target therefrom
generate a multispectral reflectance data cube of the target from the received images of the target”.
Darty teaches in the same field of endeavor, a plurality of light emitters (0113 describes for example, referring to FIG. 4, an imaging device (e.g., hyperspectral/multispectral camera) capable of illuminating an object at eight wavelengths can have two light sources 104-4 and 104-12), wherein the illumination system and hyperspectral imaging system are cooperatively configured to illuminate a target with a sequence of different lighting illuminations (0094 describes sequentially illuminating the object with narrowband light sources of various wavelengths and capturing images of the object) and image the target during each of the different lighting illuminations in the sequence (0091 describes the systems can capture true two-dimensional co-axial images by sequentially resolving images of the object at different wavelengths using respective optical detectors in a plurality of optical detectors), wherein the hyperspectral processing system is operably coupled with the hyperspectral imaging system and configured to receive images of the target therefrom and generate a multispectral reflectance data cube of the target from the received images of the target (0136 describes each of the optical detectors 206 used to resolve images for assembly into a hyperspectral/multispectral data cube; 0235 describes the hyperspectral/multispectral imaging system comprises a display 1304 which receives an image (e.g., a color image, mono-wavelength image, or hyperspectral/multispectral image); 0212 describes a data processing software module 1334 for manipulating an acquired image or set of images, a hyperspectral/multispectral data cube data store 1335 for storing hyperspectral/ multispectral data cubes cube 1336 assembled from a plurality of hyperspectral/multispectral data planes). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Darty into the invention of Jones in order to configure the capsule hyperspectral system e.g. as claimed because it works to improve the imaging efficiency (Darty 0096 describes the use of matching illumination and detection elements also eliminates wasted light in bands other than those desired for imaging, thereby improving the efficiency of the imaging system).
Jones does not further disclose:
“a tether having a capsule end coupled to the imaging capsule and a system end coupled to the hyperspectral processing system”. Freeman teaches in the same field of endeavor, a tether having a capsule end coupled to the imaging capsule and a system end coupled to the hyperspectral processing system (3:32-34 describes medical devices comprising a capsule that can be swallowed by a patient and positioned to a target tissue; 8:50-54 describes in working with hyperspectral data or other imaging or other spectral data. The cap 302 may be tethered to the sheath (by tethers inside the balloon) to allow the endoscope to rotate while inside the balloon 104). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Freeman into the invention of Jones in order to configure the capsule hyperspectral system e.g. as claimed in order to improve the positioning and reliability of performance (Freeman: 7:30-35 describes the advantage of this invention is the replacement of the push of a traditional endoscope down the esophagus (which could result in tearing) with the device being pulled by the swallowing motion and the body's peristalsis, and provides more diagnostic capability in terms of choice of location for imaging than a capsule).
“wherein the tether is communicatively coupled with the hyperspectral imaging system and hyperspectral processing system so as to pass data therebetween”. Tearney teaches in the same field of endeavor, wherein the tether is communicatively coupled with the hyperspectral imaging system and hyperspectral processing system so as to pass data therebetween (0088; FIG. 19 shows data that was collected from a duodenum region of a patient using a tethered OCT capsule system with a conventional pill-shaped capsule having a single outer diameter).
“wherein the capsule hyperspectral system is configured to use a 60 Hz frame rate or greater to minimize motion artifacts during screen capture”. Jones does not specify the frame rate of the device. Ohno teaches in the same field of endeavor, providing a frame rate of 60 fps for a spectroscopic device at 0183. Since Jones fails to disclose the nature of the frame rate, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable frame rate known in the art, including the one taught by Ohno to achieve the predictable result of hyperspectral imaging. Additionally, capturing at 60 fps provides additional information than for example at 30 fps which is described at 0181 and 0183.
Modified Jones discloses for claim 7, “The capsule hyperspectral system of claim 1, wherein the light emitters comprise light-emitting diodes (LEDs), wherein the illumination system comprises at least three LEDs having at least three different emission wavelengths (Darty: the illumination system comprises at least three LEDs having at least three different color bands (see Table 2 different colors: 0118 light sources emitting radiation in the ultraviolet spectrum (wavelengths from about 10 nm to about 400 nm), visible spectrum (wavelengths from about 400 nm to about 760 nm); 0129 and thus the semiconducting material used to form the LED. Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2).
Modified Jones discloses for claim 8, “The capsule hyperspectral system of claim 7, wherein at least one LED is a white light LED (Darty: at least one LED is a white light LED 0129 Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2)”.
Modified Jones discloses for claim 11, “The capsule hyperspectral system of claim 7, wherein the illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different color bands (Darty: 0113 illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different color bands, for example, referring to FIG. 4, an imaging device (e.g., hyperspectral/multispectral camera) capable of illuminating an object at eight wavelengths can have two light sources (104-4 and 104-12), positioned symmetrically about the objective lens 106, which emit radiation having the same wavelength(s), e.g., two identical narrowband LEDs. Likewise, four lights (104-4, 104-8, 104-12, and 104-16), emitting radiation of the same wavelength(s), can be positioned symmetrically about the objective lens)”.
Modified Jones discloses for claim 85, “The capsule hyperspectral system of claim 1, wherein the illumination system and the hyperspectral imaging system are cooperatively configured to: illuminate the target with a first lighting illumination; image the target during the first lighting illumination; illuminate the target with a second lighting illumination; image the target during the second lighting illumination; illuminate the target with a third lighting illumination; and image the target during the third lighting illumination (fig 16, 0037 describes three illumination sources in the context of hyperspectral imaging 0288)”.
Modified Jones discloses for claim 88, “The capsule hyperspectral system of claim 85, wherein the first lighting illumination, the second lighting illumination, and the third lighting illumination each includes illumination by at least two LEDs (Darty: 0015 at least two LEDs are colored LEDs with different color bands, the broadband light source is a white light-emitting diode; 0128 LEDs reach maximum intensities very rapidly, for example a red LED can achieve full brightness in under a microsecond, allowing for faster cycling through a plurality of illumination wavelengths)”.
Modified Jones discloses for claim 91 (as provided for claim 7), “The capsule hyperspectral system of claim 1, wherein the illumination system comprises at least three LEDs and a CMOS-based imaging sensor, the capsule hyperspectral system being capable to provide hyperspectral decomposition of high-resolution images (Darty: the illumination system comprises at least three LEDs having at least three different color bands (see Table 2 different colors: 0118 light sources emitting radiation in the ultraviolet spectrum (wavelengths from about 10 nm to about 400 nm), visible spectrum (wavelengths from about 400 nm to about 760 nm); 0129 and thus the semiconducting material used to form the LED. Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2)”.
Modified Jones discloses for claim 92, “The capsule hyperspectral system of claim 91, wherein the at least three LEDs have at least three different color emission wavelengths; and wherein at least one LED is a white light LED (Darty: 0113 illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different color bands, for example, referring to FIG. 4, an imaging device (e.g., hyperspectral/multispectral camera) capable of illuminating an object at eight wavelengths can have two light sources (104-4 and 104-12), positioned symmetrically about the objective lens 106, which emit radiation having the same wavelength(s), e.g., two identical narrowband LEDs. Likewise, four lights (104-4, 104-8, 104-12, and 104-16), emitting radiation of the same wavelength(s), can be positioned symmetrically about the objective lens)”.
Modified Jones discloses for claim 93, “The capsule hyperspectral system of claim 85, wherein the illumination system comprises at least three LEDs having at least three different color emission wavelengths, and a CMOS-based imaging sensor, the capsule hyperspectral system being capable to provide hyperspectral decomposition of high-resolution images, and wherein at least one LED is a white light LED (Darty: the illumination system comprises at least three LEDs having at least three different color bands (see Table 2 different colors: 0118 light sources emitting radiation in the ultraviolet spectrum (wavelengths from about 10 nm to about 400 nm), visible spectrum (wavelengths from about 400 nm to about 760 nm); 0129 and thus the semiconducting material used to form the LED. Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2)”.
Modified Jones discloses for claim 95, “The capsule hyperspectral system of claim 94, wherein the illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different emission wavelengths (Darty: 0113 illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different color bands, for example, referring to FIG. 4, an imaging device (e.g., hyperspectral/multispectral camera) capable of illuminating an object at eight wavelengths can have two light sources (104-4 and 104-12), positioned symmetrically about the objective lens 106, which emit radiation having the same wavelength(s), e.g., two identical narrowband LEDs. Likewise, four lights (104-4, 104-8, 104-12, and 104-16), emitting radiation of the same wavelength(s), can be positioned symmetrically about the objective lens)”.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jones, Darty, Freeman, Tearney, and Ohno as applied to claim 7 above, and further in view of Weber US20020147383 .
Jones does not disclose for claim 12, “The capsule hyperspectral system of claim 7, wherein an emission wavelength of each LED is selected such that a white and/or pinkish surface on healthy tissue and a red surface on non-healthy tissue can be visible identified and distinguished from each other”. Weber teaches in the same field of endeavor, malignant tissue appears more red with respect to benign tissue with illumination selective for sensitivity to malignant lesions with respect to benign tissue (0063). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Weber into the invention of Jones in order to configure the capsule e.g. as claimed because allows more accurate, true-to-life depiction of tissue type with respect to color (0063).
Claim(s) 86, 87, 96-99 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jones, Darty, Freeman, Tearney, and Ohno as applied to claim 85 above, and further in view of Cutrale US20190287222 .
Modified Jones does not disclose for claim 86, “The capsule hyperspectral system of claim 85, wherein the illumination system and the hyperspectral imaging system are cooperatively configured to: illuminate a reference target with a first lighting illumination; image the reference target during the first lighting illumination; illuminate the reference target with a second lighting illumination that is different than the first lighting illumination; image the reference target during the second lighting illumination; illuminate the reference target with a third lighting illumination that is different from the first lighting illumination and second lighting illumination; and image the reference target during the third lighting illumination”. Cutrale teaches in the same field of endeavor, a hyperspectral imaging system that can separate overlapping colors or spectra in an image and then display the result as an “unmixed” color image that uses a reference image for calibration (0058, 0062). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Cutrale into the invention of Jones in order to configure the system e.g. as claimed, specifically imaging using a reference because provides for a manner to calibrate the system (0058).
Modified Jones discloses for claim 87, “The capsule hyperspectral system of claim 86, wherein the reference target comprises a color standard image (Cutrale: 0058 describes the reference image may be any reference material wherein unmixed color image of the reference material is determined prior to the generation of unmixed color image of the target. For example, the reference material may be a physical structure, a chemical molecule, a biological molecule, a biological activity (e.g. physiological change) as a result of physical structural change and/or disease)”.
Modified Jones discloses for claim 96, “The capsule hyperspectral system of claim 1, wherein the at least one imaging sensor has a configuration that is configured to: detect target electromagnetic radiation that is at least one of: absorbed, transmitted, refracted, reflected, or emitted, by at least one physical point on the target, wherein the detected target electromagnetic radiation comprises at least two target waves, each target wave having an intensity and a unique wavelength; detect the intensity and the wavelength of each target wave; transmit the detected target electromagnetic radiation, and each target wave's detected intensity and wavelength to the hyperspectral processing system; form a target image of the target using the detected target electromagnetic radiation, wherein the target image comprises at least two pixels, and wherein each pixel corresponds to one physical point on the target; form at least one intensity spectrum for each pixel using the detected intensity and wavelength of each target wave; generate the multispectral reflectance data cube from the at least one intensity spectrum for each pixel; transform the formed intensity spectrum of each pixel using a Fourier transform into a complex-valued function based on the intensity spectrum of each pixel, wherein each complex-valued function has at least one real component and at least one imaginary component; use only a first harmonic and/or a second harmonic (Cutrale: 0054 which describes using at least one harmonic) of the Fourier transform to generate an unmixed color image of the target apply a denoising filter on both the real component and the imaginary component of each complex-valued function at least once to produce a denoised real value and a denoised imaginary value for each pixel; form one phasor point on a phasor plane for each pixel by plotting the denoised real value against the denoised imaginary value of each pixel; map back the phasor point to a corresponding pixel on the target image based on a geometric position of the phasor point on the phasor plane; assign an arbitrary color to the corresponding pixel based on the geometric position of the phasor point on the phasor plane; and generate an unmixed color image of the target based on the assigned arbitrary color (Cutrale: 0053 describes this particular function with corresponding related paragraphs to describe additional details)”.
Modified Jones discloses for claim 97, “The capsule hyperspectral system of claim 96, wherein the hyperspectral processing system is configured to form the unmixed color image of the target at a signal- to- noise ratio of the at least one spectrum in a range of 1.2 to 50 (Cutrale: 0056 describes the exact range)”.
Modified Jones discloses for claim 98, “The capsule hyperspectral system of claim 96, wherein the hyperspectral processing system is configured to form the unmixed color image of the target at a signal- to- noise ratio of the at least one spectrum in a range of 2.0 to 50 (Cutrale: 0056 describes the exact range)”.
Modified Jones discloses for claim 99, “The capsule hyperspectral system of claim 96, wherein the hyperspectral processing system is configured to: illuminate the target with a first lighting illumination; image the target during the first lighting illumination; illuminate the target with a second lighting illumination; image the target during the second lighting illumination; illuminate the target with a third lighting illumination (fig 16, 0037 describes three illumination sources in the context of hyperspectral imaging 0288); and image the target during the third lighting illumination, wherein the illumination system comprises at least three LEDs having at least three different color emission wavelengths (Darty: the illumination system comprises at least three LEDs having at least three different color bands (see Table 2 different colors: 0118 light sources emitting radiation in the ultraviolet spectrum (wavelengths from about 10 nm to about 400 nm), visible spectrum (wavelengths from about 400 nm to about 760 nm); 0129 and thus the semiconducting material used to form the LED. Non-limiting examples of semiconductor materials that can be used to produce an LED emitting light at a specific wavelength are provided in Table 2), and a non-specialized CMOS-based imaging sensor such that software-based hyperspectral decomposition of high-resolution images is enabled, wherein at least one LED is a white light LED (Darty: 0113 illumination system comprises at least six LEDs that include at least two white light LEDs and at least four colored LEDs with at least two different color bands, for example, referring to FIG. 4, an imaging device (e.g., hyperspectral/multispectral camera) capable of illuminating an object at eight wavelengths can have two light sources (104-4 and 104-12), positioned symmetrically about the objective lens 106, which emit radiation having the same wavelength(s), e.g., two identical narrowband LEDs. Likewise, four lights (104-4, 104-8, 104-12, and 104-16), emitting radiation of the same wavelength(s), can be positioned symmetrically about the objective lens)”.
Claim(s) 89 and 94 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jones, Darty, Freeman, Tearney, and Ohno as applied to claim 85 above, and further in view of Gala US20220358755 .
Jones does not disclose for claim 89, “The capsule hyperspectral system of claim 85, wherein the hyperspectral processing system has a configuration that obtains a spectrum for each pixel of the images and generates a transformation matrix from the spectrum of each pixel”. Jones does not specify the particular analysis method. Gala teaches in the same field of endeavor, a multi-stage pseudo-inverse method, as illustrated in FIGS. 2A and 2B, can be used to reconstruct a hyperspectral cube from digital images, the known spectral reflectance factors 104 of the color standard can be used to solve for a transformation matrix 108 (0093). Since Jones fails to disclose the nature of the analysis method, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable method known in the art, including the one taught by Gala to achieve the predictable result of hyperspectral imaging.
Modified Jones discloses for claim 94 (as provided for in claim 89), “The capsule hyperspectral system of claim 85, wherein the capsule hyperspectral system is configured to obtain data comprising three sets of encoding information from three sets of illumination and imaging to prepare a transformation matrix such that the transformation matrix allows for reconstructing the imaged target (Gala: 0093).
Claim(s) 90 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jones, Darty, Freeman, Tearney, and Ohno as applied to claim 85 above, and further in view of Ye US20160241797.
Jones does not disclose for claim 90, “The capsule hyperspectral system of claim 85, wherein the multispectral reflectance data cube is obtained from a pseudo-inverse method with the received images of the target”. Jones does not specify the particular method of analysis. Ye teaches in the same field of endeavor, a multispectral light imaging camera system which specifically utilizes a classical pseudo inverse method (0073) with respect to hyperspectral data cube analysis (0066). Since Jones fails to disclose the nature of the analysis method, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable method known in the art, including the one taught by Ye to achieve the predictable result of hyperspectral imaging.
Response to Arguments
Applicant has not provided any arguments with respect to the prior art rejections.
Multiple attempts to reach the attorney Jason Croft for a potential examiner’s amendment to advance prosecution were not returned.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANHTUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795
07/01/26
Prosecution Insights