DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 23 February 2026 has been entered.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 18/160,364 (the instant application), filed on 27 January 2023.
Response to Arguments
Claims 1, 8, and 9 are amended. Claim 5 is canceled. Claim 10 is added. Claims 1-3 and 6-10 are pending in this action.
Applicant's arguments, pg. 8-14, filed 23 February 2026 regarding the rejection of claims 1-4 and 5-9 under 35 U.S.C. 103 have been fully considered and are partially persuasive. The applicant argues individually that Gou et al. (WO 2021230230 A1; applicant referred to as '230; hereafter, Gou), Hu et al. (CN 104933672 A; hereafter, Hu), Hartov et al. (WO 2012068042 A2; applicant referred to as '042; hereafter, Hartov), Kruecker et al. (CN 112055870 A; applicant referred to as '820; hereafter, Kruecker), and Hu et al. (“MR to Ultrasound Registration . . .” full reference in PTO-892; applicant referred to as Hu; hereafter, Ahmed) do not disclose any of the amended claim language of claim 1. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Therefore, these arguments are not persuasive. Additionally, Guo was not relied upon during the previous rejection and therefore has no bearing on the present application.
Applicant did present a sufficient argument with regard to Jendoubi et al. (“Top-Down Approach . . .” full reference in PTO-892; hereafter, Jendoubi) and Assaf et al. (EP 1837828 B1; applicant referred to as '828; hereafter, Assaf).
Applicant's arguments, pg. 10-11, filed 23 February 2026, argue that Jendoubi particularly does not disclose a parameter indicating a degree of unevenness of an edge. The examiner disagrees. When considering the term "a parameter indicating the unevenness of an edge", the examiner must give the term its broadest reasonable interpretation. The examiner interprets the term to mean a parameter which changes value based on unevenness of an edge of a target organ. Unevenness is understood as sharp or sudden changes in the edge as opposed to smooth, gradual changes. In the teaching of Jendoubi, Jendoubi discloses a vector field GVF which is based on a minimization of an energy function including edge gradient map ∇f which has vectors pointing to the image edges, see Jendoubi pg. 3 col. 1 para. 1. As a gradient of the edge, ∇f is understood to indicate changes in the curvature of the edge. This understanding is based on the common function of a gradient. Since GVF is a vector field minimizing that gradient, it is understood to fit the vectors to the edge as best as possible. Therefore, the GVF vector field indicates areas of change in curvature by the values of the vectors. This is exemplified by Fig. 3(e) which shows that the vectors concentrate on areas of sharp change in curvature of the edge, i.e. an unevenness, see the regions indicated by the arrows below. Therefore, the applicant’s argument is not persuasive.
PNG
media_image1.png
192
176
media_image1.png
Greyscale
Further regarding Jendoubi, the applicant argues that Jendoubi does not disclose "calculating and using a parameter indicating a degree of unevenness of that edge", see applicants remarks pg. 11. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). When considered in combination with other references, it would be obvious to combine the deformation calculation of Jendoubi with a reference using a deformation calculation for registration because doing so would amount to a simple substitution of one known element for another with predictable results. See the rejection under 35 U.S.C. 103 for the complete motivation to combine Jendoubi with other references to disclose the subject matter of claim 1.
Applicant’s arguments, see pg. 12-14, filed 23 February 2023, with respect to the teachings of Assaf have been fully considered and are persuasive. Specifically, the applicant provides evidence showing the weight wi of Assaf does not disclose a weight based on a degree of deformation, as the amendments to the claim language clarifies. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Lorenz et al. (US 20150161791 A1; hereafter, Lorenz).
Lorenz discloses:
assign, for each of the plurality of position, a weight to a similarity between the medical images at each of the plurality of positions ([0035] "Generally, the RSF [registration steering factor] term allows for down (or up) weighting the contribution of the image similarity term D for different voxels or regions of voxels of the image. " Therefore, similarity is assigned a weight. As it is assigned a weight "for different voxels or regions of voxels of the image" it is understood to assign a weight at each of the plurality of positions), based on the degree of deformation at each of the plurality of positions ([0042]-[0043] deformation fields between two images are determined and used in calculating E(x). [0045] RSF, i.e. weight, is generated based on E(x), therefore, weight is based on deformation), such that a position having a smaller degree of deformation is assigned a larger weight and a position having a larger degree of deformation is assigned a smaller weight ([0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight);
calculate a weighted similarity between the medical images using the similarity at each of the plurality of position and the assigned weights (per the applicant's specification pg. 15 para. 2, "weighted similarity is obtained by giving a large weight to an area with a small deformation and giving a small weight to an area with a large deformation." Therefore, the examiner understands weighted similarity as the combination of similarity and weight as described in the previous limitations making this limitation a summarizing limitation without further limiting the claim. [0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight. This is understood as weighted similarity),
and perform a registration so as to optimize the weighted similarity as an objective function ([0027] "The image registration algorithm 204 includes a non-rigid registration that optimizes an objective function that includes an image similarity term and a regularization term." See also [0057] for application of the objective function to a method which considers deformation, through E(x), in registering images. While this is for a non-rigid registration, a person of ordinary skill would be equipped to combine this with the rigid registration of Symon because doing so represents a simple substitution of one known element, the non-rigid registration of Lorenz, for another known element, the rigid registration of Symon, to obtain predictable results, registration between two images),
Therefore, claim 1 is rejected under 35 U.S.C. 103. Claims 8 and 9 were similarly amended and are similarly rejected. The full rejection, including motivations to combine, are included below in the section "Claim Rejections - 35 USC § 103".
Claim Objections
Claims 1, 8, and 9 are objected to because of the following informalities:
Claim 1, delete the comma between "positions" and "based" on line 10. The current placement of the comma separates "based on the degree of deformation" from "assign a weight to a similarity" causing confusion on whether the assigning is based on the degree of deformation or if some other item is based on the degree of deformation.
Claim 8, delete the comma between "positions" and "based" on line 8. See the reasoning for claim 1 above.
Claim 9, delete the comma between "positions" and "based" on line 9. See the reasoning for claim 1 above.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Symon et al. (U.S. Publ. No. 20160143576; hereafter, Symon) in view of Lorenz et al. (US 20150161791 A1; hereafter, Lorenz) in further view of Jendoubi et al. (“Top-Down Approach . . .” full reference in PTO-892; hereafter, Jendoubi) and of Hartov et al. (WO 2012068042 A2; hereafter, Hartov).
Regarding claim 1, Symon discloses:
extract a registration target area ([0045] step d), the ultrasound is segmented, i.e. a target area is extracted) corresponding to a registration target organ ([0045] step d), the segmented area corresponds to the prostate. In this example the prostate is the target organ) from an ultrasonic image by acquiring edges of the registration target organ from the ultrasonic image ([0045] step d), the ultrasound image is what is segmented. The segmentation produces a contour surface of the prostate. The contour surface of the prostate is understood as acquiring edges of the organ from the image) among a plurality of types of medical images including the registration target organ ([0045] step (a) an MRI scan of the prostate is received. [0045] step c), an ultrasound scan of the prostate is received. Therefore, there is a plurality of types of medical images, MRI and ultrasound, including the target organ),
calculate a degree of deformation of a plurality of positions in the registration target area ([0045] step d), the contour is comprised of a plurality of landmark points. The landmark points are understood as the plurality of positions in the target area. [0045] step f, a matching cost is calculated between landmark points. The matching cost is further described in [0068]-[0070] and Fig. 2, the matching cost describes a hypothetical cost of matching particular landmark points by counting the number of matches in a region. The greater the deformation between the shapes the higher the matching cost would be due to less matches in a region. Therefore, the matching cost can be understood as a degree of deformation),
and perform a rigid registration on the plurality of types of medical images ([0045] step g), a linear transformation is performed to match the landmark point of the MRI to the ultrasound image) by adjusting a rotation amount and a translation amount between the medical images (this limitation is describing the commonly understood definition of a rigid registration, namely performing registration by adjusting rotation and translation between images. Therefore, this limitation does not further limit the claim from the previous limitation which claims rigid registration)
Symon does not disclose expressly assigning a weight to a similarity between images based on the degree of deformation, calculating a weighted similarity, and registering by optimizing an objective function.
Lorenz discloses:
A medical image processing apparatus comprising: processing circuitry configured to ([0024] the apparatus comprises one or more processors which are understood to include processing circuitry)
assign, for each of the plurality of position, a weight to a similarity between the medical images at each of the plurality of positions ([0035] "Generally, the RSF [registration steering factor] term allows for down (or up) weighting the contribution of the image similarity term D for different voxels or regions of voxels of the image. " Therefore, similarity is assigned a weight. As it is assigned a weight "for different voxels or regions of voxels of the image" it is understood to assign a weight at each of the plurality of positions), based on the degree of deformation at each of the plurality of positions ([0042]-[0043] deformation fields between two images are determined and used in calculating E(x). [0045] RSF, i.e. weight, is generated based on E(x), therefore, weight is based on deformation), such that a position having a smaller degree of deformation is assigned a larger weight and a position having a larger degree of deformation is assigned a smaller weight ([0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight);
calculate a weighted similarity between the medical images using the similarity at each of the plurality of position and the assigned weights (per the applicant's specification pg. 15 para. 2, "weighted similarity is obtained by giving a large weight to an area with a small deformation and giving a small weight to an area with a large deformation." Therefore, the examiner understands weighted similarity as the combination of similarity and weight as described in the previous limitations making this limitation a summarizing limitation without further limiting the claim. [0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight. This is understood as weighted similarity),
and perform a registration so as to optimize the weighted similarity as an objective function ([0027] "The image registration algorithm 204 includes a non-rigid registration that optimizes an objective function that includes an image similarity term and a regularization term." See also [0057] for application of the objective function to a method which considers deformation, through E(x), in registering images. While this is for a non-rigid registration, a person of ordinary skill would be equipped to combine this with the rigid registration of Symon because doing so represents a simple substitution of one known element, the non-rigid registration of Lorenz, for another known element, the rigid registration of Symon, to obtain predictable results, registration between two images),
Lorenz is combinable with Symon because it is from the same field of endeavor of registering medical images (Symon, [0001]; Lorenz, [0001]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the weighting based registration of Lorenz with the invention of Symon.
The motivation for doing so would have been "improving a result of a non-rigid registration by making the non-rigid registration less sensitive to imperfect image information by employing a registration steering factor with the non-rigid registration, wherein the non-rigid registration includes an optimization of an image similarity term and a regularization term" (Lorenz, [0009]).
Therefore, it would have been obvious to combine Lorenz with Symon.
Symon in view of Lorenz does not disclose expressly a parameter indicating the unevenness of an edge.
Jendoubi discloses:
wherein the degree of deformation includes at least one of a parameter indicating an unevenness of an edge (Pg. 3, col. 1, para. 2, a vector field, GVF, defines characteristics of the edges of the prostate. Fig. 3(e), the image shows how the direction of the vectors change with the curvature of the edge of the prostate. This is understood to indicate the unevenness of an edge by indicating changes in the edge),
Jendoubi is combinable with Symon in view of Lorenz because it is from the related field of endeavor of segmenting a prostate from an ultrasound image (Jendoubi, pg. 1, col. 2, para. 3).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the unevenness of an edge parameter of Jendoubi with the invention of Symon in view of Lorenz.
The motivation for doing so would have been to counter weaknesses of traditional method which has "poor convergence to concave boundaries" such as those which occur due to probe pressure (Jendoubi, pg. 3, col. 1, para. 1). Further, doing so represents substituting a known element, the unevenness of an edge of Jendoubi, for another known element, the deformation field of Symon in view of Lorenz, to obtain predictable results, a registration based on deformation which specifically corrects "convergence to concave boundaries" (Jendoubi, pg. 3, col. 1, para. 1).
Therefore, it would have been obvious to combine Jendoubi with Symon in view of Lorenz.
Symon in view of Lorenz in further view of Jendoubi does not disclose expressly a parameter indicating a distance from an organ to a pressing object.
Hartov discloses:
wherein the degree of deformation includes at least one of a parameter indicating a distance from an organ to a pressing object (Pg. 13 of 25, para. 1, an image is dilated based on the relative distance from the ultrasound probe, therefore, the distance from the pressing object to the organ is a parameter describing the deformation),
Hartov is combinable with Symon in view of Lorenz in further view of Jendoubi because it is from the same field of endeavor of registering ultrasound images with other modalities (Hartov, pg. 3 of 25, "Field of Invention").
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the parameter indicating the distance from an organ to a pressing object of Hartov with the invention of Symon in view of Lorenz in further view of Jendoubi.
The motivation for doing so would have been because using this parameter allows for "manipulations . . . based on a predetermined set of parameters depending upon the target organ type, or, alternatively, may be optimized based on conditions particular to a specific US image" (Hartov, pg. 13 of 25, para. 1).
Therefore, it would have been obvious to combine Hartov with Symon in view of Lorenz in further view of Jendoubi to obtain the invention as specified in claim 1.
Regarding claim 2, Symon in view of Lorenz in further view of Jendoubi and Hartov discloses the subject matter claim 1.
Symon further discloses:
wherein the processing circuitry is further configured to cause a display to: display a fusion image representing a registration result that is a result of the rigid registration ([0091] and Fig. 3, the final fused image is displayed),
Symon does not disclose expressly to display registration reliability of different positions in the registration result.
Lorenz discloses:
and display registration reliability of different positions in the registration result on the basis of the similarity of the positions in the fusion image ([0023] "For example, where the image registration apparatus 118 determines, based on the decision threshold, that an image quality of registered images satisfies the threshold, the image registration apparatus 118 can present a notification that the registration is acceptable. However, where the image registration apparatus 118 determines, based on the decision threshold, that an image quality of a voxel and/or region of voxels of one or more of the registered images does not satisfy the threshold, the image registration apparatus 118 can present a notification indicating so and/or a recommendation to perform another registration of the images," The display of a notification indicating whether the quality is acceptable is understood as a "registration reliability").
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the display of registration reliability of Lorenz with the invention of Symon.
The motivation for doing so would have been that "the image registration apparatus 118 can present a notification indicating so and/or a recommendation to perform another registration of the images, steering the registration way from the image similar term for the voxel and/or region of voxels" (Lorenz, [0023]).
Therefore, it would have been obvious to combine Lorenz with Symon to obtain the invention as specified in claim 2.
Regarding claim 3, Symon in view of Lorenz in further view of Jendoubi and Hartov discloses the subject matter of claim 1.
Symon further discloses:
wherein each position in the positions is a sub-area including one pixel point or a plurality of pixel points in the registration target area ([0045] the target area is described as a "contour comprising a plurality of 3D landmark points". "landmark points" are understood to comprise a pixel point in the target area, i.e. the contour).
Regarding claim 8, Symon discloses:
A medical image processing method, comprising: extracting a registration target area ([0045] step d), the ultrasound is segmented, i.e. a target area is extracted) corresponding to a registration target organ ([0045] step d), the segmented area corresponds to the prostate. In this example the prostate is the target organ) from an ultrasonic image by acquiring edges of the registration target organ from the ultrasonic image ([0045] step d), the ultrasound image is what is segmented. The segmentation produces a contour surface of the prostate. The contour surface of the prostate is understood as acquiring edges of the organ from the image) among a plurality of types of medical images including the registration target organ ([0045] step (a) an MRI scan of the prostate is received. [0045] step c), an ultrasound scan of the prostate is received. Therefore, there is a plurality of types of medical images, MRI and ultrasound, including the target organ),
calculating a degree of deformation of a plurality of positions in the registration target area ([0045] step d), the contour is comprised of a plurality of landmark points. The landmark points are understood as the plurality of positions in the target area. [0045] step f, a matching cost is calculated between landmark points. The matching cost is further described in [0068]-[0070] and Fig. 2, the matching cost describes a hypothetical cost of matching particular landmark points by counting the number of matches in a region. The greater the deformation between the shapes the higher the matching cost would be due to less matches in a region. Therefore, the matching cost can be understood as a degree of deformation),
and performing a rigid registration on the plurality of types of medical images ([0045] step g), a linear transformation is performed to match the landmark point of the MRI to the ultrasound image) by adjusting a rotation amount and a translation amount between the medical images (this limitation is describing the commonly understood definition of a rigid registration, namely performing registration by adjusting rotation and translation between images. Therefore, this limitation does not further limit the claim from the previous limitation which claims rigid registration)
Symon does not disclose expressly assigning a weight to a similarity between images based on the degree of deformation, calculating a weighted similarity, and registering by optimizing an objective function.
Lorenz discloses:
assigning, for each of the plurality of position, a weight to a similarity between the medical images at each of the plurality of positions ([0035] "Generally, the RSF [registration steering factor] term allows for down (or up) weighting the contribution of the image similarity term D for different voxels or regions of voxels of the image. " Therefore, similarity is assigned a weight. As it is assigned a weight "for different voxels or regions of voxels of the image" it is understood to assign a weight at each of the plurality of positions), based on the degree of deformation at each of the plurality of positions ([0042]-[0043] deformation fields between two images are determined and used in calculating E(x). [0045] RSF, i.e. weight, is generated based on E(x), therefore, weight is based on deformation), such that a position having a smaller degree of deformation is assigned a larger weight and a position having a larger degree of deformation is assigned a smaller weight ([0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight);
calculating a weighted similarity between the medical images using the similarity at each of the plurality of position and the assigned weights (per the applicant's specification pg. 15 para. 2, "weighted similarity is obtained by giving a large weight to an area with a small deformation and giving a small weight to an area with a large deformation." Therefore, the examiner understands weighted similarity as the combination of similarity and weight as described in the previous limitations making this limitation a summarizing limitation without further limiting the claim. [0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight. This is understood as weighted similarity),
and performing a registration so as to optimize the weighted similarity as an objective function ([0027] "The image registration algorithm 204 includes a non-rigid registration that optimizes an objective function that includes an image similarity term and a regularization term." See also [0057] for application of the objective function to a method which considers deformation, through E(x), in registering images. While this is for a non-rigid registration, a person of ordinary skill would be equipped to combine this with the rigid registration of Symon because doing so represents a simple substitution of one known element, the non-rigid registration of Lorenz, for another known element, the rigid registration of Symon, to obtain predictable results, registration between two images),
Lorenz is combinable with Symon because it is from the same field of endeavor of registering medical images (Symon, [0001]; Lorenz, [0001]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the weighting based registration of Lorenz with the invention of Symon.
The motivation for doing so would have been "improving a result of a non-rigid registration by making the non-rigid registration less sensitive to imperfect image information by employing a registration steering factor with the non-rigid registration, wherein the non-rigid registration includes an optimization of an image similarity term and a regularization term" (Lorenz, [0009]).
Therefore, it would have been obvious to combine Lorenz with Symon.
Symon in view of Lorenz does not disclose expressly a parameter indicating the unevenness of an edge.
Jendoubi discloses:
wherein the degree of deformation includes at least one of a parameter indicating an unevenness of an edge (Pg. 3, col. 1, para. 2, a vector field, GVF, defines characteristics of the edges of the prostate. Fig. 3(e), the image shows how the direction of the vectors change with the curvature of the edge of the prostate. This is understood to indicate the unevenness of an edge by indicating changes in the edge),
Jendoubi is combinable with Symon in view of Lorenz because it is from the related field of endeavor of segmenting a prostate from an ultrasound image (Jendoubi, pg. 1, col. 2, para. 3).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the unevenness of an edge parameter of Jendoubi with the invention of Symon in view of Lorenz.
The motivation for doing so would have been to counter weaknesses of traditional method which has "poor convergence to concave boundaries" such as those which occur due to probe pressure (Jendoubi, pg. 3, col. 1, para. 1). Further, doing so represents substituting a known element, the unevenness of an edge of Jendoubi, for another known element, the deformation field of Symon in view of Lorenz, to obtain predictable results, a registration based on deformation which specifically corrects "convergence to concave boundaries" (Jendoubi, pg. 3, col. 1, para. 1).
Therefore, it would have been obvious to combine Jendoubi with Symon in view of Lorenz.
Symon in view of Lorenz in further view of Jendoubi does not disclose expressly a parameter indicating a distance from an organ to a pressing object.
Hartov discloses:
wherein the degree of deformation includes at least one of a parameter indicating a distance from an organ to a pressing object (Pg. 13 of 25, para. 1, an image is dilated based on the relative distance from the ultrasound probe, therefore, the distance from the pressing object to the organ is a parameter describing the deformation),
Hartov is combinable with Symon in view of Lorenz in further view of Jendoubi because it is from the same field of endeavor of registering ultrasound images with other modalities (Hartov, pg. 3 of 25, "Field of Invention").
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the parameter indicating the distance from an organ to a pressing object of Hartov with the invention of Symon in view of Lorenz in further view of Jendoubi.
The motivation for doing so would have been because using this parameter allows for "manipulations . . . based on a predetermined set of parameters depending upon the target organ type, or, alternatively, may be optimized based on conditions particular to a specific US image" (Hartov, pg. 13 of 25, para. 1).
Therefore, it would have been obvious to combine Hartov with Symon in view of Lorenz in further view of Jendoubi to obtain the invention as specified in claim 8.
Regarding claim 9, Symon discloses:
extracting a registration target area ([0045] step d), the ultrasound is segmented, i.e. a target area is extracted) corresponding to a registration target organ ([0045] step d), the segmented area corresponds to the prostate. In this example the prostate is the target organ) from an ultrasonic image by acquiring edges of the registration target organ from the ultrasonic image ([0045] step d), the ultrasound image is what is segmented. The segmentation produces a contour surface of the prostate. The contour surface of the prostate is understood as acquiring edges of the organ from the image) among a plurality of types of medical images including the registration target organ ([0045] step (a) an MRI scan of the prostate is received. [0045] step c), an ultrasound scan of the prostate is received. Therefore, there is a plurality of types of medical images, MRI and ultrasound, including the target organ),
calculating a degree of deformation of a plurality of positions in the registration target area ([0045] step d), the contour is comprised of a plurality of landmark points. The landmark points are understood as the plurality of positions in the target area. [0045] step f, a matching cost is calculated between landmark points. The matching cost is further described in [0068]-[0070] and Fig. 2, the matching cost describes a hypothetical cost of matching particular landmark points by counting the number of matches in a region. The greater the deformation between the shapes the higher the matching cost would be due to less matches in a region. Therefore, the matching cost can be understood as a degree of deformation),
and performing a rigid registration on the plurality of types of medical images ([0045] step g), a linear transformation is performed to match the landmark point of the MRI to the ultrasound image) by adjusting a rotation amount and a translation amount between the medical images (this limitation is describing the commonly understood definition of a rigid registration, namely performing registration by adjusting rotation and translation between images. Therefore, this limitation does not further limit the claim from the previous limitation which claims rigid registration)
Symon does not disclose expressly assigning a weight to a similarity between images based on the degree of deformation, calculating a weighted similarity, and registering by optimizing an objective function.
Lorenz discloses:
A non-transitory computer readable medium comprising instructions that cause a computer to execute ([0024] a non-transitory memory stores instructions to be executed by a processor):
assigning, for each of the plurality of position, a weight to a similarity between the medical images at each of the plurality of positions ([0035] "Generally, the RSF [registration steering factor] term allows for down (or up) weighting the contribution of the image similarity term D for different voxels or regions of voxels of the image. " Therefore, similarity is assigned a weight. As it is assigned a weight "for different voxels or regions of voxels of the image" it is understood to assign a weight at each of the plurality of positions), based on the degree of deformation at each of the plurality of positions ([0042]-[0043] deformation fields between two images are determined and used in calculating E(x). [0045] RSF, i.e. weight, is generated based on E(x), therefore, weight is based on deformation), such that a position having a smaller degree of deformation is assigned a larger weight and a position having a larger degree of deformation is assigned a smaller weight ([0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight);
calculating a weighted similarity between the types of the medical images using the similarity at each of the plurality of position and the assigned weights (per the applicant's specification pg. 15 para. 2, "weighted similarity is obtained by giving a large weight to an area with a small deformation and giving a small weight to an area with a large deformation." Therefore, the examiner understands weighted similarity as the combination of similarity and weight as described in the previous limitations making this limitation a summarizing limitation without further limiting the claim. [0044] and [0047], a region with low E(x) is associated with low error or low deformation and has a high RSF, i.e. larger weight, and a region with high E(x) is associated with higher error or high deformation and has a low RSF, i.e. smaller weight. This is understood as weighted similarity),
and performing a registration so as to optimize the weighted similarity as an objective function ([0027] "The image registration algorithm 204 includes a non-rigid registration that optimizes an objective function that includes an image similarity term and a regularization term." See also [0057] for application of the objective function to a method which considers deformation, through E(x), in registering images. While this is for a non-rigid registration, a person of ordinary skill would be equipped to combine this with the rigid registration of Symon because doing so represents a simple substitution of one known element, the non-rigid registration of Lorenz, for another known element, the rigid registration of Symon, to obtain predictable results, registration between two images),
Lorenz is combinable with Symon because it is from the same field of endeavor of registering medical images (Symon, [0001]; Lorenz, [0001]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the weighting based registration of Lorenz with the invention of Symon.
The motivation for doing so would have been "improving a result of a non-rigid registration by making the non-rigid registration less sensitive to imperfect image information by employing a registration steering factor with the non-rigid registration, wherein the non-rigid registration includes an optimization of an image similarity term and a regularization term" (Lorenz, [0009]).
Therefore, it would have been obvious to combine Lorenz with Symon.
Symon in view of Lorenz does not disclose expressly a parameter indicating the unevenness of an edge.
Jendoubi discloses:
wherein the degree of deformation includes at least one of a parameter indicating an unevenness of an edge (Pg. 3, col. 1, para. 2, a vector field, GVF, defines characteristics of the edges of the prostate. Fig. 3(e), the image shows how the direction of the vectors change with the curvature of the edge of the prostate. This is understood to indicate the unevenness of an edge by indicating changes in the edge),
Jendoubi is combinable with Symon in view of Lorenz because it is from the related field of endeavor of segmenting a prostate from an ultrasound image (Jendoubi, pg. 1, col. 2, para. 3).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the unevenness of an edge parameter of Jendoubi with the invention of Symon in view of Lorenz.
The motivation for doing so would have been to counter weaknesses of traditional method which has "poor convergence to concave boundaries" such as those which occur due to probe pressure (Jendoubi, pg. 3, col. 1, para. 1). Further, doing so represents substituting a known element, the unevenness of an edge of Jendoubi, for another known element, the deformation field of Symon in view of Lorenz, to obtain predictable results, a registration based on deformation which specifically corrects "convergence to concave boundaries" (Jendoubi, pg. 3, col. 1, para. 1).
Therefore, it would have been obvious to combine Jendoubi with Symon in view of Lorenz.
Symon in view of Lorenz in further view of Jendoubi does not disclose expressly a parameter indicating a distance from an organ to a pressing object.
Hartov discloses:
wherein the degree of deformation includes at least one of a parameter indicating a distance from an organ to a pressing object (Pg. 13 of 25, para. 1, an image is dilated based on the relative distance from the ultrasound probe, therefore, the distance from the pressing object to the organ is a parameter describing the deformation),
Hartov is combinable with Symon in view of Lorenz in further view of Jendoubi because it is from the same field of endeavor of registering ultrasound images with other modalities (Hartov, pg. 3 of 25, "Field of Invention").
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the parameter indicating the distance from an organ to a pressing object of Hartov with the invention of Symon in view of Lorenz in further view of Jendoubi.
The motivation for doing so would have been because using this parameter allows for "manipulations . . . based on a predetermined set of parameters depending upon the target organ type, or, alternatively, may be optimized based on conditions particular to a specific US image" (Hartov, pg. 13 of 25, para. 1).
Therefore, it would have been obvious to combine Hartov with Symon in view of Lorenz in further view of Jendoubi to obtain the invention as specified in claim 9.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Symon et al. (U.S. Publ. No. 20160143576; hereafter, Symon) in view of Lorenz et al. (US 20150161791 A1; hereafter, Lorenz) in further view of Jendoubi et al. (“Top-Down Approach . . .” full reference in PTO-892; hereafter, Jendoubi) and of Hartov et al. (WO 2012068042 A2; hereafter, Hartov) in further view of Kruecker et al. (CN 112055870 A; hereafter, Kruecker).
Regarding claim 6, Symon in view of Lorenz in further view of Jendoubi and of Hartov discloses the subject matter of claim 2.
Symon in view of Lorenz in further view of Jendoubi and of Hartov does not disclose expressly that causing the display to display the registration reliability presents a level of the reliability in different colors for different positions in the fusion image.
Kruecker discloses:
wherein, when causing the display to display the registration reliability, the processing circuitry is further configured to present a level of the registration reliability in different colors for different positions in the fusion image ([0039] "The user may be notified by a visual warning that varies based on the degree to which the image registration fails the passing assessment. For example, the color or intensity of the electronic display may vary based on the degree to which the images failed to register." The statement that the color may vary is understood to teach different colors. [0039] "The display may also indicate the location within the fusion image where a significant misalignment . . . is detected" is understood to teach different positions in the image).
Kruecker is combinable with Symon in view of Lorenz in further view of Jendoubi and of Hartov because it is from the same field of endeavor of registering two images together (Kruecker, [0006]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the display of registration reliability with colors of Kruecker with the invention of Symon in view of Lorenz in further view of Jendoubi and of Hartov.
The motivation for doing so would have been that “the user is notified that the evaluation result of the image registration is unqualified” (Kruecker, [0039]).
Therefore, it would have been obvious to combine Kruecker with Symon in view of Lorenz in further view of Jendoubi and of Hartov to obtain the invention as specified in claim 6.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Symon et al. (U.S. Publ. No. 20160143576; hereafter, Symon) in view of Lorenz et al. (US 20150161791 A1; hereafter, Lorenz) in further view of Jendoubi et al. (“Top-Down Approach . . .” full reference in PTO-892; hereafter, Jendoubi) and of Hartov et al. (WO 2012068042 A2; hereafter, Hartov) in further view of Hu et al. (CN 104933672 B; hereafter, Hu).
Regarding claim 7, Symon in view of Lorenz in further view of Jendoubi and of Hartov discloses the subject matter of claim 1.
While Lorenz discloses a non-rigid registration (see at least Lorenz, [0027]), Symon in view of Lorenz in further view of Jendoubi and of Hartov does not disclose expressly performing a non-rigid registration after the rigid registration at the positions to generate deformation vector fields, corrects the degree of deformation on the basis of magnitudes of vectors in the deformation field and performs rigid registration again on the types of medical images.
Hu discloses:
wherein the processing circuitry is further configured to perform, after the rigid registration is performed, a non-rigid registration on the types of the medical images ([0015] during process (6), the ultrasound image is transformed according to a non-rigid deformation field and registered with the CT image. Transforming by the non-rigid deformation field and registering is understood as a non-rigid registration. This occurs after the rigid registration of [0011]. The registration being performed on the ultrasound and CT images is understood as the types of medical images. The details of process (6) begin at [0038])
at the positions to generate deformation vector fields ([0013] the process (4) calculates a current non-rigid deformation field. [0026] in process (4), the deformation field is calculated by "the deformation of each point" which is understood to include the points selected in [0019], i.e. positions. [0026] The deformation field includes calculation of vectors, for instance see the use of the vector transpose T, and therefore is understood as a deformation vector field),
correct the degree of deformation of the positions based on magnitudes of vectors in the deformation vector fields corresponding to the types of medical images ([0014] process (5) corrects the current deformation field to an optimal deformation field. This is done on the basis of magnitudes of vectors because: [0026] the deformation u(x) is composed of u1(x), u2(x), u3(x) (see [0025]) which represent the deformation of each point in the x, y, z, directions. The deformation is understood to comprise a magnitude. Each point is understood to comprise the positions. Formula on pg. 20 under the paragraph labeled as [0022] in original document, the deformation field u(x) depends on a difference between c(x) and r(x). [0025] c(x) and r(x) depend on a CT image and ultrasound image respectively. Therefore, the magnitudes of the vectors in the deformation field correspond to the types (CT and ultrasound in this instance) of medical images) so that as a magnitude of the vector increases, the degree of deformation increases (Formula on pg. 20 under the paragraph labeled as [0022] in the original document, since u(x), the deformation field, is dependent on the difference of c(x) and r(x), then as the magnitude of the vector in u(x) increases, it is due to the deformation, i.e. the difference between the images, also increasing),
and perform the rigid registration again on the types of the medical images ([0040] the images are registered again. See [0020] for the first registration) based on the degree of deformation for each position corrected at the positions ([0039] the deformation field is updated, i.e. corrected. As the deformation field comprises each point it is understood to include the positions. [0040] the ultrasound image is deformed per the deformation field prior to the registering, which is understood to mean that the registering is on the basis of the corrected field) and the similarity between the types of the medical images at the positions ([0040] the ultrasound image is deformed according to the deformation field, which is understood to minimize the difference between the images, see the minimum value of h [0031] and [0036] which is the optimal deformation field [0030]. After deforming the ultrasound image to minimize difference, the images are understood to have high similarity between images and the registration is based on their similarity. That similarity would include the positions in the target area [0019]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to combine the second registration of Hu with the invention of Symon in view of Lorenz in further view of Jendoubi and of Hartov.
The motivation for doing so would have been that "a reasonable model is proposed, and a corresponding fast algorithm is designed to solve the problem" (Hu, [0042]).
Therefore, it would have been obvious to combine Hu with Symon in view of Lorenz in further view of Jendoubi and of Hartov to obtain the invention as specified in claim 7.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 10, Symon in view of Lorenz in further view of Jendoubi and of Hartov discloses the subject matter of claim 1. The closest prior art does not disclose the specific relationship described in claim 10. In reviewing the language of claim 10, the examiner recognizes that the subject matter of claim 10 appears to be claiming matter similar to equation 5 and equation 6 from the applicant's specification on pg. 15 and 16.
The claim as a whole is found non-obvious over the prior art including:
wherein the processing circuitry is configured to determine the weighted similarity as a sum, over each position of the plurality of positions, of the similarity at the position multiplied by an inverse of the calculated degree of deformation at the position.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kadir et al., US 20160140716 A1, discloses a system performing medical image registration which determines a weighting of images based on differences.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA B CROCKETT whose telephone number is (571)270-7989. The examiner can normally be reached Monday-Thursday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA B. CROCKETT/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661