DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 15 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over IDS provided reference US 2014/0046341 to DiCarlo (hereinafter DiCarlo) in view of US 2006/0039690 to Steinberg et al. (hereinafter Steinberg).
Regarding independent claim 1, DiCarlo teaches an apparatus comprising:
one or more processors (one or more processors execute method 400, see par. [0032]); and
memory storing executable instructions that, when executed by the one or more processors, cause the apparatus (instructions stored in memory, see par. [0032]) to:
identify, within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame (step 404 identifies the region of interest, and hence the rest is background, see par. [0035]);
determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region (in step 406, the luminance of the ROI is determined, since the exposure value represents the luminance of the region, the auto-exposure value is determined, see par. [0038]);
determine, based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value (in step 406, the luminance determined for the ROI will be used as the frame luminance, since the exposure value represents the luminance of the frame, the frame auto-exposure value is determined, see par. [0038]); and
adjust, based on the frame auto-exposure value, one or more auto-exposure parameters used by the image capture system to capture an additional image frame (in step 410 the exposure is adjusted in response to the luminance value, see par. [0039]).
But DiCarlo fails to clearly specify “wherein the identifying the signal region and the background region includes:
identifying pixels of the image frame that have auto-exposure values that exceed an auto-exposure value threshold to be included within the signal region, and
identifying pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region”.
However, Steinberg teaches in a second embodiment, foreground pixels in a flash image are initially determined at step 596 as those with exposure levels greater than the upper exposure threshold value VH and background pixels in a flash image are those with exposure levels less than the lower exposure threshold value VL, (see par. [0042]).
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to exposure control.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above camera taught by DiCarlo, by incorporating the teaching taught by Steinberg.
One of ordinary skill in the art would have been motivated to do this modification in order to enhance depth of field, to enhance or eliminate the background altogether, or to extract objects such as faces or people from an image as suggested by Steinberg (see par. [0008]).
Regarding claim 3, DiCarlo in view of Steinberg teaches the apparatus of claim 1, wherein the determining the one or more of the signal auto-exposure value or the background auto-exposure value comprises one or more of:
determining the signal auto-exposure value to be an average of the auto-exposure values of the pixels identified to be included within the signal region (uses average values, see Steinberg par. [0043-0044]); or
determining the background auto-exposure value to be an average of the auto-exposure values of the pixels identified to be included within the background region (uses average values, see Steinberg par. [0043-0044]).
Regarding claim 4, DiCarlo in view of Steinberg teaches the apparatus of claim 1, wherein:
the instructions, when executed by the one or more processors, cause the apparatus to determine, based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value; and
the adjusting the one or more auto-exposure parameters is based on the frame auto-exposure value (in step 410 the exposure is adjusted in response to the luminance value, see DiCarlo par. [0039]).
Regarding claim 15, DiCarlo in view of Steinberg teaches the apparatus of claim 1, wherein the one or more auto-exposure parameters include one or more of:
an exposure time parameter;
a shutter aperture parameter;
an illumination intensity parameter; or
a luminance gain parameter (parameters such as exposure time, variable aperture and light intensity are considered, see DiCarlo par. [0040]).
Regarding independent claim(s) 20, claim(s) is/are drawn to the method used by the corresponding apparatus in claim(s) 1 and is/are rejected for the same reasons used above.
Claim(s) 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over DiCarlo in view of Steinberg as applied to claim 1 above, and further in view of IDS provided reference US 2003/0078477 to Kang et al. (hereinafter Kang).
Regarding claim 13, DiCarlo in view of Steinberg teaches the apparatus of claim 1, but fails to clearly specify
“wherein: the image frame depicts fluorescence content against a darkened background, the fluorescence content generated by a fluorescence imaging agent that fluoresces when illuminated by a fluorescence illumination source;
the signal region identified within the image frame corresponds to the fluorescence content; and
the background region identified within the image frame corresponds to the darkened background”.
However, Kang discloses that a conventional fluorescence endoscope system comprises illumination source for viewing internal organs after injecting a contrast agent into the body in order to view the region, see par. [0006].
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to endoscope system for medical use.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above system, as taught by DiCarlo in view of Steinberg, by combining components, as taught by Kang.
One of ordinary skill in the art would have been motivated to do this modification in order to observe specific regions of the body as suggested by Kang (see par. [0006]).
Regarding claim 14, DiCarlo in view of Steinberg and Kang teaches the apparatus of claim 13, wherein:
the image capture system includes an endoscopic image capture device configured to capture the image frame as part of an image frame sequence that depicts a view of a body undergoing a fluorescence-guided medical procedure (see Kang par. [0006]); and
the endoscopic image capture device captures the image frame while operating in a fluorescence imaging mode that facilitates viewing tissue of the body to which the fluorescence imaging agent has been applied (see Kang par. [0006]).
Regarding independent claim 16, DiCarlo in view of Steinberg and Kang as combined above teaches a system (corresponding to the apparatus in claim 1) comprising:
a fluorescence illumination source configured to illuminate tissue within a body undergoing a fluorescence-guided medical procedure (see Kang par. [0006], as discussed above),
wherein a portion of the tissue includes a fluorescence imaging agent that fluoresces when illuminated by the fluorescence illumination source (see Kang par. [0006] as discussed above);
an image capture device configured to capture an image frame sequence that includes an image frame depicting a view of the body as the tissue is illuminated by the fluorescence illumination source (see Kang par. [0006] as discussed above); and
one or more processors configured to (one or more processors execute method 400, see DiCarlo par. [0032]):
identify, within the image frame captured by the image capture device, a signal region of the image frame associated with the portion of the tissue that includes the fluorescence imaging agent and a background region of the image frame associated with a portion of the tissue that does not include the fluorescence imaging agent (step 404 identifies the region of interest, and hence the rest is background, see DiCarlo par. [0035], as combined with Kang the fluorescence imaging is performed);
determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region (in step 406, the luminance determined for the ROI will be used as the frame luminance, since the exposure value represents the luminance of the frame, the frame auto-exposure value is determined, see DiCarlo par. [0038]);
adjust, based on one or more of the signal auto-exposure value or the background auto-exposure value, one or more auto-exposure parameters used by the image capture device or the fluorescence illumination source to capture an additional image frame of the image frame sequence (in step 410 the exposure is adjusted in response to the luminance value, see par. [0039]).
wherein the identifying the signal region and the background region includes:
identifying pixels of the image frame that have auto-exposure values that exceed an auto-exposure value threshold to be included within the signal region (foreground pixels in a flash image are initially determined at step 596 as those with exposure levels greater than the upper exposure threshold value VH, see Steinberg par. [0042]) and
identifying pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region (background pixels in a flash image are those with exposure levels less than the lower exposure threshold value VL, see Steinberg par. [0042]).
Regarding claim 17, DiCarlo in view of Steinberg and Kang teaches the system of claim 16, wherein: the one or more processors are further configured to determine, based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value; and
the adjusting the one or more auto-exposure parameters is based on the frame auto-exposure value (in step 410 the exposure is adjusted in response to the luminance value, see DiCarlo par. [0039]).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-17 and 19 of U.S. Patent No. 12,087,014.
Although the claims at issue are not identical, they are not patentably distinct from each other because claims of the instant application are anticipated by patent claims as shown infra.
Instant Application
Patent 12,087,014
1. An apparatus comprising:
one or more processors; and
memory storing executable instructions that, when executed by the one or more processors, cause the apparatus to:
identify, within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame; determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region; and
adjust, based on one or more of the signal auto-exposure value or the background auto-exposure value, one or more auto-exposure parameters used by the image capture system to capture an additional image frame;
wherein the identifying the signal region and the background region includes:
identifying pixels of the image frame that have auto-exposure values that exceed an auto-exposure value threshold to be included within the signal region, and identifying pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
16. A system comprising:
a fluorescence illumination source configured to illuminate tissue within a body undergoing a fluorescence-guided medical procedure, wherein a portion of the tissue includes a fluorescence imaging agent that fluoresces when illuminated by the fluorescence illumination source;
an image capture device configured to capture an image frame sequence that includes an image frame depicting a view of the body as the tissue is illuminated by the fluorescence illumination source; and one or more processors configured to: identify, within the image frame captured by the image capture device, a signal region of the image frame associated with the portion of the tissue that includes the fluorescence imaging agent and a background region of the image frame associated with a portion of the tissue that does not include the fluorescence imaging agent;
determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region; and
adjust, based on one or more of the signal auto-exposure value or the background auto-exposure value, one or more auto-exposure parameters used by the image capture device or the fluorescence illumination source to capture an additional image frame of the image frame sequence;
wherein the identifying the signal region and the background region includes:
identifying pixels of the image frame that have auto-exposure values that exceed an auto-exposure value threshold to be included within the signal region, and identifying pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
20. A method comprising:
identifying, within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame;
determining one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region; and
adjusting, based on one or more of the signal auto-exposure value or the background auto-exposure value, one or more auto-exposure parameters used by the image capture system to capture an additional image frame;
wherein the identifying the signal region and the background region includes:
identifying pixels of the image frame that have auto-exposure values that exceed an auto-exposure value threshold to be included within the signal region, and identifying pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
1. An apparatus comprising:
one or more processors; and
memory storing executable instructions that, when executed by the one or more processors, cause the apparatus to:
identify, within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame; determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region;
determine, based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value; and
adjust, based on the frame auto-exposure value, one or more auto-exposure parameters used by the image capture system to capture an additional image frame;
wherein the identifying the signal region and the background region includes: comparing auto-exposure values of pixels of the image frame to an auto-exposure value threshold; and based on the comparing,
identifying: pixels of the image frame that have auto-exposure values that exceed the auto-exposure value threshold to be included within the signal region, and pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
15. A system comprising:
a fluorescence illumination source configured to illuminate tissue within a body undergoing a fluorescence-guided medical procedure, wherein a portion of the tissue includes a fluorescence imaging agent that fluoresces when illuminated by the fluorescence illumination source;
an image capture device configured to capture an image frame sequence that includes an image frame depicting a view of the body as the tissue is illuminated by the fluorescence illumination source; and one or more processors configured to: identify, within the image frame captured by the image capture device, a signal region of the image frame associated with the portion of the tissue that includes the fluorescence imaging agent and a background region of the image frame associated with a portion of the tissue that does not include the fluorescence imaging agent;
determine one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region;
determine, based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value; and
adjust, based on the frame auto-exposure value, one or more auto-exposure parameters used by the image capture device or the fluorescence illumination source to capture an additional image frame of the image frame sequence;
wherein the identifying the signal region and the background region includes:
comparing auto-exposure values of pixels of the image frame to an auto-exposure value threshold; and
based on the comparing,
identifying: pixels of the image frame that have auto-exposure values that exceed the auto-exposure value threshold to be included within the signal region, and pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
19. A method comprising:
identifying, by a computing device and within an image frame captured by an image capture system, a signal region of the image frame and a background region of the image frame;
determining, by the computing device, one or more of a signal auto-exposure value of the signal region or a background auto-exposure value of the background region;
determining, by the computing device and based on one or more of the signal auto-exposure value or the background auto-exposure value, a frame auto-exposure value; and
adjusting, by the computing device and based on the frame auto-exposure value, one or more auto-exposure parameters used by the image capture system to capture an additional image frame;
wherein the identifying the signal region and the background region includes:
comparing auto-exposure values of pixels of the image frame to an auto-exposure value threshold; and
based on the comparing,
identifying: pixels of the image frame that have auto-exposure values that exceed the auto-exposure value threshold to be included within the signal region, and pixels of the image frame that have auto-exposure values that do not exceed the auto-exposure value threshold to be included within the background region.
Claim 2 of the instant application is encompassed by patent claim 2.
Claim 3 of the instant application is encompassed by patent claim 3.
Claim 4 of the instant application is encompassed by patent claim 1.
Claim 5 of the instant application is encompassed by patent claim 4.
Claim 6 of the instant application is encompassed by patent claim 5.
Claim 7 of the instant application is encompassed by patent claim 6.
Claim 8 of the instant application is encompassed by patent claim 7.
Claim 9 of the instant application is encompassed by patent claim 8.
Claim 10 of the instant application is encompassed by patent claim 9.
Claim 11 of the instant application is encompassed by patent claim 12.
Claim 12 of the instant application is encompassed by patent claim 13.
Claim 13 of the instant application is encompassed by patent claim 10.
Claim 14 of the instant application is encompassed by patent claim 11.
Claim 15 of the instant application is encompassed by patent claim 14.
Claim 17 of the instant application is encompassed by patent claim 15.
Claim 18 of the instant application is encompassed by patent claim 16.
Claim 19 of the instant application is encompassed by patent claim 17.
Allowable Subject Matter
Claims 2, 5-12 and 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasize that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the apparatus of claim 1, wherein the instructions, when executed by the one or more processors, cause the apparatus to dynamically determine the auto-exposure value threshold by linearly scaling a baseline threshold with a frame auto-exposure gain associated with the image frame.
Regarding claim(s) 11-12, claim(s) depend from claim 2 and is/are allowable for the same reasons stated above.
Regarding claim 5, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasize that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the apparatus of claim 4, wherein:
the instructions, when executed by the one or more processors, cause the apparatus to determine a signal region size value representative of a size of the signal region within the image frame; and
the determining the frame auto-exposure value is further based on the signal region size value.
Regarding claim(s) 6-10, claim(s) depend from claim 5 and is/are allowable for the same reasons stated above.
Regarding claim 18, none of the prior art cited alone or in combination provides the motivation to teach the following claimed limitations, with emphasize that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements make them allowable over the prior art of record, the system of claim 17, wherein the one or more processors are configured to: determine one or more of a signal auto-exposure target of the signal region or a background auto-exposure target of the background region; determine a signal region size value representative of a size of the signal region within the image frame; and determine, based on the signal region size value and one or more of the signal auto-exposure target or the background auto-exposure target, a frame auto-exposure target; wherein: the determining the frame auto-exposure value is further based on the signal region size value, and the adjusting the one or more auto-exposure parameters is further based on the frame auto-exposure target.
Regarding claim(s) 19, claim(s) depend from claim 18 and is/are allowable for the same reasons stated above.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2007/0110427 to Tsai teaches a method of exposure compensation comprises capturing a first image without a flash, capturing a second image with the flash, generating a F-number, a shutter speed value, and a flash value according to brightness distribution of the first and second images, and capturing a third image according to the F-number, the shutter speed value, and the flash value.
US 2012/0105668 to VELARDE et al. teaches an example image capture device determines a region of interest using a first image captured while a light source is powered off and a second image captured while a light source is powered on and uses the region of interest to automatically set configurations. In one example, an image capture device includes a controlled light source, an image sensor configured to capture images, and a processing unit configured to cause the image sensor to capture a first image of a scene while the controlled light source is powered off, cause the image sensor to capture a second image of the scene while the controlled light source is powered on, calculate luminance differences between a plurality of regions in the first image and a plurality of collocated regions in the second image, and determine that a region of interest includes those regions for which the luminance differences exceed a threshold.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGEL L GARCES-RIVERA whose telephone number is (571)270-7268. The examiner can normally be reached Mon-Fri 9AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 571-727-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANGEL L GARCES-RIVERA/Examiner, Art Unit 2637
/SINH TRAN/Supervisory Patent Examiner, Art Unit 2637