DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are currently pending in U.S. Patent Application No. 18/419,528 and an Office action on the merits follows.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the grounds of nonstatutory double patenting as being unpatentable and/or obvious over one or more claims of:
1) U.S. Patent No. 11,880,978 to parent Application No. 17/361,315 - CON of
2) U.S. Patent No. 11,049,254 to parent Application No. 16/410,119 - CON of
3) U.S. Patent No. 10,290,108 to parent application 15/638,327.
Although the claims at issue are not identical, they are not patentably distinct from each other because claims of reference anticipate and/or render obvious independent claim(s) of the instant application. Reference may be made to Double Patenting rejections and corresponding limitation mapping tables as found in the Non-Final Office Actions for each of the parent applications listed above, all commonly assigned and limited by means of Terminal Disclaimer(s) in view of corresponding predecessor(s). Specifically that Non-Final mailed 09/01/2020 for Application 16/410,119 and Non-Final Office Action mailed 01/03/2023 for Application 17/361,315.
In the interest of compact prosecution, Examiner requests the same/Terminal Disclaimer for the instant application. The conflicting claims are not patentably distinct from each other for the following reasons:
• Instant claims and claims of reference recite common subject matter, and recite the open ended transitional phrase “comprising” which does not preclude any additional elements recited by claims of reference;
• Language/terminology of instant claim(s) constituting minor/slight variations from the claims of reference, if/where present, require interpretations under Broadest Reasonable Interpretation and/or plain meaning definitions (MPEP 2173 and 2111) equivalent to/met by language of the reference claims in view of that corresponding/shared Specification. While the disclosure of reference may not be used as prior art (Double Patenting concerns the claims of reference), portions of the specification which provide support for reference claims may also be examined and considered when addressing the scope of claim(s) of reference and the issue of whether an instant claim defines an obvious variation or falls within the scope of an invention claimed in the claim(s) of reference. See MPEP 804 with reference to In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 9-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 9 recites the limitation "the predetermined point in the characteristic curve" at line 6 (final line). There is insufficient antecedent basis for this limitation in the claim. Basis for the limitation in question is established in claim 6, however claim 9 depends directly on claim 5, not claim 6.
Dependent claims 10-11 inherit and fail to cure that deficiency identified above for intervening claim 9, and are rejected accordingly.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
1. Claims 1, 13, 19 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Heinlein et al. (US 8,428,324 B2) (cited in Applicant’s 4/11/2024 IDS).
As to claim 1, Heinlein discloses an image processing method implemented on at least one machine each of which has at least one processor and storage (col 1 lines 30-37 “A mammographic imaging system may be coupled to a workstation, for example, a PACS (Picture Archiving and Communication System) workstation, on which a mammogram may be viewed and reviewed by a physician or other medical professionals”), the method comprising:
acquiring a target image, the target image including a plurality of elements, an element corresponding to a pixel or a voxel (col 4 lines 5-15 “FIG. 1 illustrates a summary flow diagram of a method 100 for processing digital mammograms, identified in FIG. 1 as raw mammograms GR. The method 100 may be used on several different imaging platforms, including direct digital platforms, digitized film platforms, and computed radiography (CR) film platforms. A raw mammogram contains original raw image data, captured by an imaging system. The raw mammogram does not include any processing of the raw image data”, col 1 lines 20-25 “Digital mammographic images or mammograms are usually presented as gray scale images having individual pixels, with each pixel having a pixel value corresponding to a specific gray scale value”, Fig. 4, etc.,);
decomposing the target image into a low frequency sub-image and a high frequency sub-image (Fig. 17, multi-scale decomposition of GR, col 6 lines 15-25 “The multi-scale decomposition divides the mammogram GR into a low-pass portion T, four band-pass filter portions of B1, B2, B3, B4, and a high-pass portion H. The different portions T, H, B1, B2, B3, B4 are independently modified and reconstructed to a brightness-optimized mammogram in which the brightness decline is balanced in the boundary region”, col 5 lines 55-60, etc.,);
transforming the low frequency sub-image using a gray level transformation to obtain a transformed low frequency sub-image (Fig. 15B, Fig. 2, col 6 lines 15-25 “The different portions T… are independently modified”, see Fig. 2 110-150 (110 in particular), col 4 lines 55-65 “FIG. 2 illustrates a more detailed flow diagram of the individual steps of the method of FIG.1. In a first step 110, a peripheral density equalization process is performed to remove the brightness decline along the breast edge. In this step, the brightness decline in a boundary region of the mammogram GR between the edge of the breast and the background becomes balanced. In a second step 120, a contrast adaptation process is performed between different regions of the mammogram GR, for example between the breast interior and the breast edge. In a third step 130, a global contrast optimization process is performed to optimize the brightness of the mammogram GR. In a fourth step 140, an enhancement of fine details of the mammogram GR is performed to increase the sharpness of fine structures. In a fifth step 150, a skin line image of the mammogram GR is generated. In a sixth step 160, a reconstruction of the skin line image is performed by adding the skin line image into the mammogram”, Fig. 6 for low-pass image T, col 8 lines 1-10 “This reconstructed image R' shows all details, the bright-less decline is removed, and application of window level settings is not necessary. However, the image R does not look like a typical mammogram, the musculus pectoralis 15 and dense breast tissue are not well separated from fat tissue, and there is too high contrast along the edge of the breast. In order to achieve a mammogram containing structures with a greater width, at least to a certain degree, the low pass portion T should not be eliminated, but modified by subtracting a threshold from all low pass pixels and setting all negative pixels to 0”, col 8 line 15 “FIG. 15b illustrates a gray scale pixel value figurative representation of a modified low-pass portion T ’. The low-pass portion of T is modified”, col 8 lines 15-25, Fig. 16, etc.,; Examiner understands the processing/optimization most pertinent for the case of the low frequency sub-image to be a sharpening of the edge/skinline, removal of brightness decline in a periphery of the edgeline, and/or increasing uniformity/selectively processing the interior regions in a manner accounting for e.g. differences in thickness – however any of those disclosed in Heinlein as applicable to T read); and
reconstructing, based on the transformed low frequency sub-image and the high frequency sub-image, a composite image (Fig. 17 multi-scale reconstruction based in part on T’, H, E5, etc., col 8 lines 15-25 “Before the reconstruction the modified low-pass portion T is multiplied by a scaling factor, in order to absorb its influence in relation to the band-pass filter portions of B1, B2, B3, B4 and the high-pass portion of H”, col 8 lines 25-32 “FIG. 16 illustrates the reconstructed mammogram image R including the modified low-pass portion T". FIG. 17 illustrates a Summary presentation of the steps for removing brightness decline”, col 8 lines 30-40 “In step 120, a reconciliation of the contrast between the breast interior and the breast edge is performed (contrast adaption). This step is described with reference to FIGS. 18 and 19. FIG. 18 illustrates a figurative representation of a mask M0' for determination of an interior or central region of an imaged object (breast) for contrast adaption. FIG. 19 illustrates a gray scale pixel value figurative representation of a mask M0'' for production of a modified high-pass portion for contrast adaption”, etc., ).
As to claim 13, Heinlein discloses the method of claim 1.
Heinlein further discloses the method, the high frequency sub-image including a plurality of elements, and the method further comprising transforming the high frequency sub-image, wherein transforming the high frequency sub-image (Heinlein Fig. 17 operation between H and M5 so as to produce E5 prior to reconstruction removing overshoot portions of H, generation of that ‘modified high-pass portion’, etc.,) comprises:
generating a weight image for the high frequency sub-image, the weight image including a plurality of weights corresponding to the plurality of elements (Heinlein Fig. 17 M5, col 7 lines 40-50 “After multiplying the band-pass filter portion B1, B2, B3, B4 and the high-pass portion H with the eroded mask image, the overshoot region is removed”, crossfade mask M0", etc.,); and
updating, based on the weight image, the high frequency sub-image (E5 with overshoot portions of H removed, col 7, col 9 lines 1-15 “From the original high-pass portion, a modified high-pass portion is generated with the help of a cross fade mask. Afterwards, reconstruction is accomplished. The cross fade mask M0" is produced from mask M0' by adding a certain amount of fading. The cross fade mask M0" contains values between zero and one, whereby the pixel values of the cross fade mask M0" determine the portions of the original high pass portion and the modified high-pass portion”).
As to claim 19, this claim is the system claim corresponding to the method of claim 1 and is rejected accordingly.
As to claim 20, this claim is the non-transitory CRM claim corresponding to the method of claim 1 and is rejected accordingly.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Heinlein et al. (US 8,428,324 B2) in view of Bick et al. (US 5,452,367).
As to claim 2, Heinlein discloses the method of claim 1.
Heinlein further discloses the method wherein the transforming the low frequency sub-image comprises:
determining a characteristic curve based on the low frequency sub-image (Fig. 5-6, 10a), the characteristic curve illustrating a relationship between a distance (Fig. 5-6, 10a x axis illustrating pixel distance along line I-I which extends from the interior towards the skinline/ boundary region towards the edge of the breast – see Figures) and a gray level corresponding to the distance (Fig. 5-6, 10a, etc., pixel value y-axis);
Heinlein suggests determining, based on the characteristic curve, a transformation curve, the transformation curve illustrating a relationship between the gray level before transformation (Fig. 6, Original gray level O, for which it can be seen the difference between T and O, col 6 lines 10-15)
generating the transformed low frequency sub-image by updating (Heinlein gray value transformation from T to produce T’), Fig. 2 110 peripheral density equalization to remove brightness decline along breast edge, transforming T to T’, wherein the predetermined region(s) are those associated with 12 and/or the edge/skinline distinct from 11).
While Heinlein, does illustrate/disclose both those plots/curves associated with original gray levels and those associated with T (of the low frequency sub image prior to transformation), Heinlein does not appear to illustrate any plot of T’ for example, and appears to fall silent regarding any explicitly identified transformation curve that is the basis in generating T’. Such a curve however would simply illustrate a series of functions, over distance and/or gray level intervals, describing the difference between T and T’ for the low frequency sub-image. The transformation between T and T’ is representable in such a manner given that disclosure of col 8 lines 15-25 – a median is determined from a histogram/distribution of pixel intensities, and this median is subtracted from T, wherein all resultant negative values are then set to zero (col 8 line 10 low pass portion of T is not eliminated entirely but modified by subtracting a threshold from low pass pixels).
Bick evidences the obvious nature of an image processing method implemented on at least one machine each of which has at !east one processor and storage (Fig. 15, memories 151/160, processing circuits152-159 and 161, col 5 lines 60-66, Fig. 24, etc.), the method comprising:
acquiring a target image including a plurality of elements corresponding to a plurality of pixels or voxels (Fig 22 220, col 6 lines 63-66 “Referring to FIG. 15, radiographic images of an object are obtained from an image acquisition device 150 which could be an x-ray exposure device and a laser digitizer, and input to the system. Each breast image is digitized and put into memory 151”), wherein the target image is generated by an imaging device (device 150/210) and includes a first region and a second region (e.g. Figures 13, 19 20, etc., ROIs/pixels/voxels associated with bone(s)/microcalcification(s), breast tissue, skin line/edge/contour, outside, etc., see also those ‘regions’/‘zones’ of e.g. col 4 lines 40-65), a first set of the plurality of elements are included in the first region, and a second set of the plurality of elements are included in the second region (internal/breast and edge/border/skin-line pixels/regions respectively, Figures 13, 19, col 6 lines 55 - col 7 line 5):
determining, based on the first set of the plurality of elements included in the first
region and the second set of the plurality of elements included in the second region, a characteristic curve corresponding to the target image (Figures 19 and 23, illustrating characteristic curve/function/plot of gray value vs. distance (mm) (comparable to Applicant’s Fig. 6), Figure 22 223 and 224, col 7 lines 29-45 “identification of the external skinline (step 222), the Euclidean distance for each potential breast pixel to the external skinline is calculated (step 223). Next, the average gray value as a function of distance from the external skinline is examined and used in determining the enhancement factor (step 224). This enhancement selectively enhances the peripheral region in order to simultaneously display the center of the breast and the skinline regions without loss in contrast”), wherein
the characteristic curve illustrates a relationship between a distance and a gray level corresponding to the distance (Figures 19 and 23, col 7 lines 29-45), the distance indicates a reference distance between each of the first set of the plurality of elements and a corresponding element in the second set of the plurality of elements (Fig. 23, col 7 lines 29-45 “Euclidean distance for each potential breast pixel to the external skinline”, etc.), and the gravy level indicates an average gravy level of one or more elements in the first set of the plurality of elements associated with the same reference distance (col 7 lines 29-45 “the average gray value as a function of distance from the external skinline”);
determining, based on the characteristic curve, a transformation curve (enhancement curve/factor and correction of Fig. 2 224-225, based on characteristic curve as per col 7 lines 29-45, ‘enhancement curve’ of col 7 lines 40-52 “The enhancement curve is obtained from a reversal of a fitted curve (such as a polynomial fit) to the average gray values (prior to enhancement) as a function of distance from the skinline. Constraints include the need for the fitted curve to have continuously smaller values, i.e. smaller gray values as distance increases. The values from the enhancement curve can be added to the corresponding pixels at the particular distance if the average gray value curve to produce the enhanced gray value curve. Other operations, besides addition, can also be
used”), the transformation curve illustrating a relationship between gray levels of the plurality of elements before transformation and gray levels of the plurality of elements after transformation (Fig. 23, col 7 lines 29-52); and
determining, based on the transformation curve, an updated image (enhanced/ corrected image displayed in 226 based on update/enhancement of Fig. 22 224-225, col 7 line 64 “Data are passed to the image enhancement circuit 246 in order to process the image. The processed image is then displayed on the display system 248”);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Heinlein to further comprise generating a transformation curve characterizing a difference between one or more of those sub-images prior to transformation/optimization, to include the low frequency sub-image T of Heinlein, and those versions thereof post transformation, such as T’ (Fig. 15a transformed to 15b) as taught/suggested by that comparable transformation curve of Bick, the motivation as similarly taught/suggested therein that such a transformation curve would be a convenient way of modeling a series of functions over one or more distance and/or gray level intervals, characterized by a reasonable expectation of success, and particularly suited for Heinlein which seeks to provide a mammogram optimization generic to different vendors and/or platforms (Heinlein col 2 lines 10-20) (even if such a transformation curve is used by developers to ensure an optimization suitable for one or more vendors/platforms and not necessarily presented to a radiologist during a clinical workflow).
As to claim 3, Heinlein in view of Bick teaches/suggests the method of claim 2.
Heinlein in view of Bick further teaches/suggests the method wherein the distance is a minimum distance between first elements in the low frequency sub-image and second elements in a reference edge of the low frequency sub-image (Heinlein as modified above wherein reference edge is the breast edge/skinline, boundary between 12 and 11 and the distance is the ‘periphery’ of said edge in the direction of the interior/12, see also Bick as applied above), the reference edge includes an ROI edge of the low frequency sub-image (Heinlein breast/skin edge line), and the gray level corresponding to the distance is determined as an average gray level of the first elements with the same distance (Heinlein suggests an average gray level in that dispersion measure of col 8 lines 55-65 however these are from within the high-pass portion for 120, Bick however teaches/suggests determining such a gray level corresponding to the distance, col 7 lines 40-45 “The enhancement curve is obtained from a reversal of a fitted curve (such as a polynomial fit) to the average gray values (prior to enhancement) as a function of distance from the skinline”, col 6 lines 25-30 “The potential internal skinline points are identified as a local gradient minimum within a certain distance from the outside breast contour”, col 4 lines 50-65, etc., see Bick as applied above).
As to claim 4, Heinlein in view of Bick teaches/suggests the method of claim 2.
Heinlein in view of Bick further teaches/suggests the method wherein the predetermined region includes a region within which distances of elements are within a predetermined value, and gray levels of elements in the predetermined region is close to gray levels of elements in a neighborhood region of the predetermined region (Heinlein in view of Bick as applied wherein Bick teaches/suggests a predetermined distance from the skinline for those gray values before enhancement not close to/at the desired 600 value of Fig. 23 - within a threshold distance close to the skinline (e.g. 1.25 cm) (see Fig. 23, gray value 0-200 for approximately 0-1mm, 200-400 for 1-3mm, 400-500 for 3-7mm, 500-550 for 7mm to 1cm, 550-600 for 1-1.25cm)). Heinlein further suggests the manner in which such a predetermined periphery region serves to limit the processing to those interior portions most likely to suffer brightness decline as the edgeline is approached – similar to Bick, and POSITA would recognize considering gray levels close to neighboring values ensures unwanted enhancement is not applied to those areas characterized by sufficient brightness.
2. Claims 5-11 are rejected under 35 U.S.C. 103 as being unpatentable over Heinlein et al. (US 8,428,324 B2) in view of Bick et al. (US 5,452,367) and Farrell (US 6,236,751).
As to claim 5, Heinlein in view of Bick teaches/suggests the method of claim 2.
Heinlein in view of Bick further teaches/suggests the method wherein the determining a transformation curve comprises:
dividing the characteristic curve into N characteristic curve segments (Heinlein discloses at least two segments, those that fall below and those above the median, col 8; and with reference to Fig. 4 distance based segments associated with interior region/ROI 12 as distinguished from exterior/background 11; Bick characteristic curve segmentation/ division on the basis of distance, at least in view of Fig. 19 and those ‘breast’, ‘skin’ and ‘outside’ regions, but also that 0-600 range subdivided further on a distance basis as previously identified (see remarks of Final Rejection mailed 04/28/2023 for 17/361,315); Bick Fig. 23, pre and post enhancement values suggest a transformation that differs significantly as distance from skinline increases (see amount of overlap between pre and post enhancement gray values roughly 50% between those ticks/marks on the x-axis (cm)) – serving to at least suggest N=2 segments (closer to skinline involving more transformed gray values, and further from skinline involving gray values that need not be transformed/enhanced));
determining, based on the N characteristic curve segments, N transformation curve segments, wherein a characteristic curve segment corresponds to a transformation curve segment (also suggested in Heinlein given the differences in T’ for each segment (below and above median of distribution/histogram) – below the median is set to 0; see Bick above in view of that modification/motivation as presented for the case of claim 2); and
generating, based on the N transformation curve segments, the transformation curve (Bick enhancement curve/factor and correction of Fig. 2 224-225, based on characteristic curve as per col 7 lines 29-45, ‘enhancement curve’ of col 7 lines 40-52 “The enhancement curve is obtained from a reversal of a fitted curve (such as a polynomial fit) to the average gray values (prior to enhancement) as a function of distance from the skinline. Constraints include the need for the fitted curve to have continuously smaller values, i.e. smaller gray values as distance increases. The values from the enhancement curve can be added to the corresponding pixels at the particular distance if the average gray value curve to produce the enhanced gray value curve. Other operations, besides addition, can also be used” in view of remarks above, col 7 lines 29-52; A person having ordinary skill in the art would also recognize, in view of e.g. Bick Fig. 23 wherein at least one of N segments involves minimal transformation/correction to gray values, that a plurality of curve segments may serve to simplify a transformation for a given segment, and/or eliminate entirely transformations for segments wherein the transformation would only minimally impact a final/corrected gray value).
Farrell further evidences the obvious nature of a transformation/mapping between gray values wherein a characteristic/transformation curve is segmented into N curve/ transformation segments/sections (Fig. 2, 5, etc. col 6 lines 40-55, col 7 lines 15-30, col 7 lines 49-63 “Thus, input grey value 85 is mapped to output grey value 98. As described above, a slope is determined for the first segment points between the end points of (25, 14) and (85, 98) which defines a linear transformation for the input grey values to the output grey values. Likewise, a slope is found for the second segment between end points (85, 98) and (156, 220). A tone reproduction curve is then generated based on the multiple linear transformations defined by the two segments”, col 8 lines 60-66, col 9 lines 60-66 “Therefore, the input grey space between grey levels 1 and 12 are mapped to output grey levels 14 to 90. When reproducing this image, a lot of detail is brought out in the shadow areas because the first segment range of 12 grey levels is mapped to a larger output segment range of 76 grey levels”, col 10 lines 1-10 and lines 60 – col 11 line 5). Farrell also explicitly identifies the manner in which such a segmentation serves to produce a final/aggregate transformation which differs for various segments/sections and serving to bring out levels of detail for gray level ranges of interest.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Heinlein in view of Bick so as to comprise determining N characteristic curve segments and N corresponding transformation curve segments in the generation of that enhancement/transformation curve, as taught/suggested by Bick and Farrell, the motivation as similarly taught/ suggested therein that such a piecewise transformation determination may serve to bring out a level of detail for ranges of interest and/or eliminate/minimize transformation/ computations where unnecessary, thereby improving one or more of an enhancement/ correction effect and/or efficiency with a reasonable expectation of success.
As to claim 6, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 5.
Heinlein in view of Bick and Farrell further teaches/suggests the method wherein the determining N transformation curve segments comprises: for an xth transformation curve segment of the N transformation curve segments, calculating a slope of the xth transformation curve segment based on the gray level of a predetermined point in the characteristic curve (e.g. Farrell slope determination disclosure based on those segment start and endpoints (corresponding to distances of Bick – Applicant’s P corresponding to Bick Fig. 23 where the fitted enhancement curve terminates/intersects a noise/floor gray value e.g. 50 based on illustration of Bick (or where the after enhancement curve overlaps the before enhancement curve)), e.g. col 7 lines 15-30 “based on a slope of their corresponding segment. In other words the slope of the first segment which is between points (25, 0) and (85, 104) defines a one-to-one linear transformation of the input grey values in the first segment to output grey values between 0 and 104. Similarly, the second segment defines a different slope between points (85, 104) and (156, 255) which defines a one-to-one linear transformation of the input grey values in the second segment to output grey values between 104 and 255”), a gray level of a starting point of an xth characteristic curve segment, and a gray level of an end point of the xth characteristic curve segment (e.g. Farrell col 7 lines 15-30 “defines a one-to-one linear transformation of the input grey values in the first segment to output grey values between 0 and 104”, see also those pre and post enhancement gray values of Bick Fig. 23), the xth characteristic curve segment corresponding to the xth transformation curve segment, wherein x is an integer, 1 ≤ x ≤ N (Bick as applied in Heinlein in view of Farrell comprising piecewise transformation, wherein pre and post enhancement curves correspond); and
determining the gray level of a starting point of the xth transformation curve segment (Bick as applied in Heinlein in view of Farrell determining enhancement curve values (at a particular distance/distance segment/range) corresponding to average gray values at corresponding distance prior to enhancement), comprising:
if x=1, designating the gray level of the starting point of the xth characteristic curve segment as the gray level of the starting point of the xth transformation curve segment (Bick as applied in Heinlein in view of Farrell wherein starting points of transformation and characteristic curves/segments correspond – in further view of an interpretation wherein segments are numbered in a manner such that further distances correspond to a lower/lowest index/x number); and
if 1 < x ≤ N, determining the gray level of the starting point of the xth transformation curve segment based on a gray level of a starting point of a (x-1)th transformation curve segment and a gray level variation of the (x-1)th characteristic curve segment (Bick as applied in Heinlein in view of Farrell wherein a next segment start-point corresponds to a previous segment endpoint (see Farrell disclosure above in addition to that interpretation in view of Bick Fig. 23), the N segments combined span a total distance of concern from the skinline, and a gray level variation corresponds to those minimum and maximum gray levels for a corresponding distance range/segment).
As to claim 7, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 6.
Heinlein in view of Bick and Farrell further teaches/suggests the method wherein the slope of the xth transformation curve segment is a ratio of the gray level of the predetermined point in the characteristic curve to an average gray level of the starting point and the end point of the xth characteristic curve segment (see Bick Fig. 23 as applied above – wherein the recited language borrows support from PGPUB at paragraph [0156]/Fig. 7 – illustrating a similarly shaped enhancement curve as that in Bick – see figures reproduced below).
PNG
media_image1.png
534
820
media_image1.png
Greyscale
PNG
media_image2.png
616
776
media_image2.png
Greyscale
As to claim 8, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 6.
Heinlein in view of Bick and Farrell further teaches/suggests the method wherein the gray level of the starting point of the xth transformation curve segment is a sum of the gray level of the starting point in the (x-1)th transformation curve segment and the gray level variation of the (x-1)th characteristic curve segment (see Bick as applied above, in view of that rationale as presented above in the rejection of claim(s) 2/6, and the note above for the case of claim 7, wherein claim 8 draws support from that disclosure of e.g. [0156-0158] also referencing Applicant’s Fig. 7; in other words, claims 7-8 are recited as textual descriptions of what amounts to a series of functions/equations describing the slopes, gray levels, and/or distances for those corresponding/analogous curve segments as identified in the proposed combination of prior art, with reference to Bick in particular, and are rejected accordingly).
As to claim 9, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 5.
Heinlein in view of Bick and Farrell further teaches/suggests the method comprising: determining a gray level range of the characteristic curve (Bick Fig. 23, see also range disclosure of e.g. col 4 lines 1-10, see claim 5 above), wherein the gray level range is a range within which at least one gray level is to be transformed (range associated with Heinlein breast/interior regions in the periphery of skinline/outline 2 for that processing of Heinlein 110 to remove brightness decline on approach of said edgeline, in view of Bick Fig. 23), and the gray level range corresponds to a portion of the characteristic curve (see above); and designating the maximum value or minimum value of the gray level range as the gray level of the predetermined point in the characteristic curve (Bick Figures 19 and 23 and col 7 lines 29-52 wherein the maximum value may correspond to those approx. 600 gray level values after enhancement, and/or that minimum 50 prior to enhancement, in further view of Bick disclosure suggesting different gray levels (prior to enhancement) being common based on the distance from the skinline – as does Heinlein, the brightness decline is more severe upon approaching/proximity to the skinline/edge 2).
As to claim 10, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 9.
Heinlein in view of Bick and Farrell further teaches/suggests the method wherein a count of the N characteristic curve segments is determined based on a count of gray levels greater than the minimum value of the gray level range (Heinlein as modified in view of Bick Fig. 23 wherein the N curve segments as proposed are based on the gray values between approximately 50 and 600 of Bick (along that ‘average gray values before enhancement’ curve Fig. 23) for various mm distances (see Fig. 23, gray value 0-200 for approximately 0-1mm, 200-400 for 1-3mm, 400-500 for 3-7mm, 500-550 for 7mm to 1cm, 550-600 for 1-1.25cm)). Examiner additionally notes that POSITA would recognize determining N curve segments on the basis of a count of gray levels broadly, but also within a minimum and maximum (e.g. 50 even 0, and e.g. 600 of Bick), would serve to create discrete curve segments sufficiently modeling the curve (too few segments and the slope characterizes a line that does not overlap much of the curve dipping above/below).
As to claim 11, Heinlein in view of Bick and Farrell teaches/suggests the method of claim 9.
Heinlein in view of Bick and Farrell further teaches/suggests the method wherein a count of the N characteristic curve segments is determined based on a count of gray levels lower than or equal to the maximum value of the gray level range (Bick Fig. 23 for N curve segments wherein a minimum value of a gray level range is near 0 (e.g. 50), the at least one gray level to be transformed are any of those e.g. below 600 and within a threshold distance close to the skinline (e.g. 1.25 cm (could be more for a subdivision requiring little or no enhancement)), and the ‘number of gray levels’ within that range are discrete values (along that ‘average gray values before enhancement’ curve Fig. 23) for various mm distances (see Fig. 23, gray value 0-200 for approximately 0-1mm, 200-400 for 1-3mm, 400-500 for 3-7mm, 500-550 for 7mm to 1cm, 550-600 for 1-1.25cm); see also that note regarding claim 10 above).
3. Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Heinlein et al. (US 8,428,324 B2) in view of Zhang (US 2016/0364840 A1) (cited by Applicant)
As to claim 12, Heinlein discloses the method of claim 1.
Heinlein further discloses the method wherein the decomposing comprises: decomposing, based on a first decomposition, the target image into L layers, each layer of the L layers including a low frequency sub-image and a high frequency sub-image, L ≥ 1 (see decomposition identified above, H and T/L sub-images for the case that L=1).
Heinlein further suggests a decomposition of sorts wherein for e.g. the low frequency image, the image is further divided into a low and ‘high’ versions thereof around that median/threshold, however fails to explicitly recite that second decomposing recited.
Zhang evidences the obvious nature of a pyramid/multi-level decomposition and decomposing, based on a second decomposition, the target image into L'+N image layers, each layer of the L'+N layers including a low frequency sub-image and a high frequency sub-image, L' ≥1, and N ≥ 1 (Zhang Fig. 4, 3rd level having those 4 L and H frequency pairs, [0076], [0076] “by parity of reasoning, it is possible to perform decomposition in an N-th layer. A value of N may be adjusted according to levels of the qualities of the source image and the target image”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Heinlein so as to implement a multi-level decomposition as taught/suggested by Zhang, the motivation as similarly taught/suggested therein that such a N level decomposition may enable subsequent processing such as an upscaling and/or reconstruction, that preserves high frequency details for each or a more nuanced set of bands/scale (a similar rationale is presented in Heinlein in not removing T entirely but seeking to distinguish the high and low components thereof to only remove the low – Heinlein col 8 line 1-10 – also in E1-E5, such that the proposed modification would have a more nuanced set/higher number of scales/bands).
4. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Heinlein et al. (US 8,428,324 B2).
As to claim 17, Heinlein discloses the method of claim 1.
Heinlein further discloses the method wherein the target image is a breast image, and acquiring a target image comprises:
acquiring an initial image acquired from an imaging device (Heinlein Fig. 17, GR, MO, etc.,);
extracting an ROI from the initial image, the ROI is a breast (Heinlein various images, breast region/ROI 12 as distinguished from outside/background region 11, col 7 lines 10-15 “Mask M0 in the imaged object area (breast) has pixel values set to 1 (white), while the background 11 has pixel values set to 0 (black)”, M1-5 etc.,);
extracting an ROI edge based on gray level variation characteristics of the ROI edge in the initial image, the ROI edge is a breast edge (col 7 lines 5-10 “FIG. 11 illustrates a figurative representation of a mask M0 representing a region of interest. An outline line 2 of the imaged object (breast) is shown”); and
determining the target image based on the ROI and the ROI edge (Fig. 2 110-150 based on the distinction between ROI/internal region 12 and that edge/border/skinline of 11).
While Heinlein suggests deriving M0 in what is perhaps a step performed in parallel with a step of acquiring what is drawn as the most equivalent ‘target image’ GR, it may be argued M0 is not used to derive GR itself prior to a point in which it is subject to multi-scale decomposition. It is also noted that mask M0 is utilized by Heinlein for processing primarily of those band-pass and high-pass portions/sub-images, and that a portion of processing in Heinlein is done prior to a reconstruction producing e.g. R, and this optimized/reconstructed mammogram is then processed with a subsequent decomposition into high-pass and low-pass portions (col 8 line 55, see also disclosure for decomposition of K post optimization). Accordingly Heinlein suggests a segmentation/masking prior to decomposition, even if it is not the same/first decomposition of GR. Examiner further asserts that POSITA would recognize segmentation/masking as a pre-processing step broadly to serve for data reduction purposes and remove e.g. irrelevant/ background portions (also perhaps removing pectoral muscle portions like 12a) – a teaching of Heinlein even if applied post decomposition – stated differently, it is unnecessary to decompose much of the background region regardless of the band of interest. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Heinlein in view of that very same teaching of Heinlein for determining masks generally, to mask an acquired image prior to decomposition, since such a segmentation/masking prior to decomposition may reduce computational costs associated with otherwise decomposing irrelevant e.g. background regions.
5. Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Heinlein et al. (US 8,428,324 B2) in view of Bar-Aviv et al. (US 8,605,970 B2) (cited by Applicant).
As to claim 18, Heinlein teaches/suggests the method of claim 17.
Heinlein fails to explicitly disclose any denoising as pre-processing prior to decomposition, in the acquisition of e.g. GR.
Heinlein however at the minimum suggests a gray level enhancement that is optionally applied as a form of post-processing, col 5 line 50 “The brightness-optimized mammogram may be modified by global contrast and sharpness enhancement algorithms in a comparatively easy way”, and Heinlein as proposed for the case of claim 17 above suggests a segmentation/masking (and an ‘enhanced’ image thereby) as performed prior to decomposition.
Bar-Aviv evidences the obvious nature of such a pre-processing as part of a method wherein the extracting an ROI edge based on gray level variation characteristics of the ROI edge in the initial image comprises:
denoising the initial image to generate a denoised image (“denoising the original medical image” of a) prior to that disclosed decomposition of b), col 3 lines 25-35 “a) denoising an original medical image acquired at a resolution higher than the specified resolution; and”);
enhancing gray levels of elements of the ROI edge in the denoised image to generate an enhanced image (col 8 lines 25-35 “residual image, a difference between the original and denoised image, optionally filtered to remove noise and enhance edges, is added back, fully or in part, to the denoised image”); and
extracting the ROI edge from the enhanced image (col 18 lines 1-10 “Optionally, for example in a medical image, the image is segmented into different types of tissue using any known segmentation technique, and search pixels are chosen only, or preferentially, from pixels of the same type of tissue”; BarAviv further suggests the manner in which denoising and enhancement processing prior to subsequent segmentation/related morphological processing steps may ensure those subsequently performed steps are themselves more accurate – comparable to that masking and subsequent erosion of Heinlein).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Heinlein as proposed so as to comprise denoising and gray level enhancement as pre-processing steps prior to a subsequently performed segmentation/mask generation and decomposition, as taught/suggested by BarAviv, the motivation as similarly taught/suggested therein (e.g. col 8 lines 15-25) that such a preprocessing would provide for a more accurate segmentation/mask determination, better realizing those advantages identified above for the case of claim 17, as applied to segmentation/masking prior to decomposition.
Additional References
Prior art made of record and not relied upon that is considered pertinent to applicant's disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
Allowable Subject Matter
Claims 14-16 would be allowable if rewritten to overcome (or otherwise overcome by means of Terminal Disclaimer) Double Patenting rejections set forth in this Office action (see in particular claims 7-10 of US 11,049,254 B2, and/or claims 14-16 of US 10,290,108 B2), and to include all of the limitations of the base claim and any intervening claims. References of record fail to serve in any obvious combination teaching each and every limitation as required therein. More specifically, references of record fail to identify a motivation, nor is such a motivation apparent to the Examiner more generally, as would be required for additional modification to e.g. Heinlein as applied, so as to derive that/those weight image equivalent(s) therein in the manner recited for the case of claim 14, and thereby teaching/suggesting each and every limitation as required.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN L LEMIEUX whose telephone number is (571)270-5796. The examiner can normally be reached Mon - Fri 9:00 - 6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN L LEMIEUX/Primary Examiner, Art Unit 2669