DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Arguments
Applicant’s arguments (see remarks), filed 07/10/2026, with respect to Claims 1, 3-5, 9-12, 14-16, and 18-26, have been fully considered but are respectfully unpersuasive.
On page 17, applicant argues “The Office Action relies on Li, particularly paragraphs [0013], [0135], and [0229], for the teaching that "a portion where a thickness exceeds a predetermined threshold is removed from the living tissue region." Applicant respectfully disagrees. On the contrary, para. [0229] of Li discloses a "tissue map" in which the intensity value corresponds to the thickness of calcium plaque (e.g., approximately 0.5 mm to 1.5 mm) and is displayed using color-coded indicia. This is merely a visualization technique for displaying thickness information using color; it does not remove any portion of the tissue region from the underlying data. The underlying tissue data remains intact in Li.”.
In response, the Office respectfully does not find this argument persuasive. Based on the broadest reasonable interpretation, the prior art by LI et al. (US 20200226422 A1) explicitly teaches generating region contour data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region (wherein initial lumen detection may be performed by a first neural network as described in US 9138147 B2 (incorporated by reference), which includes, for example, gradient filter such as sobel edge detection, selecting the longest contour of segments that exceed a predetermined angular/radial/elucidean distance threshold, calculating a weight based on the thickness and gap of scan lines corresponding to lumen tissue regions, and assessing whether the weight exceeds a predetermined discontinuity threshold). In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). Please also read paragraph [0013 and 0104]) in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region (Fig. 1. Paragraph [0073]-LI discloses FIGS. 16-18 show various tissue map representations generated using an OCT imaging pullback of an artery with various indicia integrated into a user interface displaying the various tissue maps to support diagnosis and treatment plans (wherein FIG. 15, for example, shows a schematic representation of various cut planes/rings/boundaries R1 to Rn that are shown along a 3D artery representation 1425 extending in the proximal and distal direction). In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425 (wherein outputted results may include masking, detected boundaries of a lumen, the thickness of luminal layers, graphical elements describing thickness such as hatching and a graphical user interface where regions can be selected, filtered and modified by class or type). Therefore, it would have been obvious to a person of ordinary skill in the art to remove a portion from an inner surface of the living tissue region facing the luminal region. LI teaches semantic segmentation, thickness and proximity measurements as well as the detection and display of individual luminal regions using thickness/distance ranges and thresholds. Moreover, LI teaches filtering, removing and reconstructing areas within the luminal region and generating proximal and distal views where a portion of living tissue facing the luminal region has been removed. Thus, this would have improved the visualizations and semantic segmentations of luminal regions as well as allowed for greater isolation of areas. Please also see US 9138147 B2 “Lumen morphology image reconstruction based on the scan line data of OCT” which is incorporated by reference in its entirety, see Fig. 2, 3C, 5, and 8 and read paragraph [0135, 0142 and 0202]).
Furthermore, in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, LI et al. (US 20200226422 A1) explicitly teaches detects, segments, and classifies the lumen, media (i.e. living tissue region on the inner surface facing the lumen), branch, plaque/calcium (i.e. non-living tissue region on the inner surface facing the lumen). LI also calculates the thickness/distance of each region and layer (please see Fig. 5l-J showing thickness measurements for the intima, media and adventitia), selects the greatest contour segment that exceeds an angular/radial/elucidean distance threshold, calculates a weight based on the thickness and gap of scan lines representing tissue regions, and assesses whether the weight exceeds a discontinuity threshold (please also see US 9138147 B2 “Lumen morphology image reconstruction based on the scan line data of OCT” which is incorporated in by reference in its entirety). Furthermore, LI projects the boundaries of luminal regions into a cut-plane regions at various distances to generate visualizations of living and non-living tissue where regions may be selected, removed, masked, filtered, restructured, highlighted and/or modified, and the thickness of luminal areas may be described by graphical elements and a range, such as hatching (please see Fig. 14-16). In addition, LI facilitates the physical ablation or removal of calcium/plaque using the tissue map and thickness results. Thus, as stated above, it would have been obvious to remove a portion from the living tissue region where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold because this would have further improved the visualizations and semantic segmentations of luminal regions as well as allowed for greater isolation of areas.
On page 17, applicant argues “Similarly, para. [0135] of Li discloses applying a "binary morphological reconstruct filter" to remove "small structures" in the carpet view image, which is a noise-removal operation. This filter operates on small noise structures in image space. In contrast, the amended claims explicitly require generating "region contour data in which a portion where a thickness exceeds a predetermined threshold is removed from the living tissue region" through the process of (i)-(v) above. This "removal" is not a display attribute or noise filtering, but rather a structural transformation of the data that produces a thin contour representation in which thick tissue portions are absent.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 17, applicant argues “In contrast, the amended claims explicitly require generating "region contour data in which a portion where a thickness exceeds a predetermined threshold is removed from the living tissue region" through the process of (i)-(v) above. This "removal" is not a display attribute or noise filtering, but rather a structural transformation of the data that produces a thin contour representation in which thick tissue portions are absent.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 17, applicant argues “Additionally, neither Buckler nor Li discloses or suggest extracting the luminal region from the classification data and generating edge data by applying an edge extraction filter to a classification extraction image generated based on the extracted luminal region as recited in claims 1, 14, and 16 as amended.”.
In response, the Office respectfully does not find this argument persuasive. Based on the broadest reasonable interpretation of the claim language, the prior art by BUCKLER et al. (US 20210390689 A1) explicitly teaches extracting the luminal region from the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0185]);
generating edge data by applying an edge extraction filter to a classification extraction image generated based on the extracted luminal region (Fig. 4. Paragraph [0169]-BUCKLER discloses systems relating to evaluating the vascular system may advantageously include/employ algorithms for evaluating vascular structure. The systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. An initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Vessel segmentation may further entail outer wall segmentation (e.g., utilizing a minimum curvature (k2) flow to account for lumen irregularities). An edge potential map is calculated as outward-downward gradients in both contrast and non-contrast. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) in a speed function to allow for median thickness in the absence of any other edge information. Ferret diameters may be employed for vessel characterization. In further embodiments, wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs);
Furthermore, in response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, BUCKLER et al. (US 20210390689 A1) applies an initial vessel segmentation including lumen segmentation. BUCKLER discloses that lumen and/or wall segmentations may be performed using CNNs and semantic segmentation. Moreover, additional confidence filters and edge potential maps may be used to further define a lumen and the inner/outer wall boundaries (please see paragraph [0169 and 0175-0176]).
On page 17, applicant argues “Li, however, does not disclose or suggest extracting the luminal region from classification data and applying an edge extraction filter to the resulting extraction image.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 17, applicant argues “Bucker does not disclose extracting only the luminal region as a separate processing step and then applying an edge extraction filter to a classification extraction image derived therefrom.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 17, applicant argues “Li, however, does not disclose or suggest extracting the luminal region from classification data and applying an edge extraction filter to the resulting extraction image.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 18, applicant argues “Accordingly, since neither Buckler nor Li, whether alone or in combination, teaches the recited extraction-followed-by-edge-extraction step (steps (i)-(ii) above), claims 1, 14, and 16 should be allowable.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
On page 18, applicant argues “Applicants submit that this application is in condition for immediate allowance, and an early notification to that effect is respectfully requested.”.
In response, the Office respectfully does not find this argument persuasive for the reasons stated above and below.
Claim Objections
Claims 22, 24 and 26 are objected to because of the following informalities:
In claim 22, Line 5, the term “generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are sent to 0;” should be changed to “generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0;” to correct a typographical error. Appropriate correction is required.
Claim 22, Line 9-10 states “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij X Mij”. The Office finds the terms “Raj = Bij X Mij” to be unclear. The Office understands that “R”, “B” and “M” to refer to the region contour matrix, the thick-line edge matrix and the mask matrix, respectively. However, it is unclear what the variables “aj” and “ij” refer to. The specification at paragraph [0078-0080] states “Element of i row and j column of the region contour matrix R is indicated by Rij. An element of i rows and j columns of the thick-line edge matrix B are indicated by Bij. An element of i row and j column of the mask matrix M are indicated by Mij”. Therefore, based on paragraph [0078-0080], the Office recommends the limitation be changed to “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, Rij = Bij X Mij, wherein i corresponds to a row number and j corresponds to a column number”. Appropriate corrections are required to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ).
Claim 24, Line 9-10 states “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij X Mij”. The Office finds the terms “Raj = Bij X Mij” to be unclear. The Office understands that “R”, “B” and “M” to refer to the region contour matrix, the thick-line edge matrix and the mask matrix, respectively. However, it is unclear what the variables “aj” and “ij” refer to. The specification at paragraph [0078-0080] states “Element of i row and j column of the region contour matrix R is indicated by Rij. An element of i rows and j columns of the thick-line edge matrix B are indicated by Bij. An element of i row and j column of the mask matrix M are indicated by Mij”. Therefore, based on paragraph [0078-0080], the Office recommends the limitation be changed to “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, Rij = Bij X Mij, wherein i corresponds to a row number and j corresponds to a column number”. Appropriate corrections are required to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ).
Claim 26, Line 9-10 states “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij X Mij”. The Office finds the terms “Raj = Bij X Mij” to be unclear. The Office understands that “R”, “B” and “M” to refer to the region contour matrix, the thick-line edge matrix and the mask matrix, respectively. However, it is unclear what the variables “aj” and “ij” refer to. The specification at paragraph [0078-0080] states “Element of i row and j column of the region contour matrix R is indicated by Rij. An element of i rows and j columns of the thick-line edge matrix B are indicated by Bij. An element of i row and j column of the mask matrix M are indicated by Mij”. Therefore, based on paragraph [0078-0080], the Office recommends the limitation be changed to “calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, Rij = Bij X Mij, wherein i corresponds to a row number and j corresponds to a column number”. Appropriate corrections are required to avoid clarity issues to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21, 23 and 25 and their associated dependent claims 22, 24, and 26 are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21, Line 1-5, recites “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region”. The Office finds the term “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque” to render the claim indefinite. The Office understands the mask as classifying regions and the mask as being opaque. However, it is unclear what “the pixel group” is being referenced for the mask’s operation given the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque. For the interest of continued prosecution and the purposes of mapping the claims, the examiner understands the limitation to be “wherein the mask makes the pixel group classified as the tissue region as opaque, the tissue region including the luminal region and the extraluminal region”. Appropriate corrections required.
Claim 23, Line 1-5, recites “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region”. The Office finds the term “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque” to render the claim indefinite. The Office understands the mask as classifying regions and the mask as being opaque. However, it is unclear what “the pixel group” is being referenced for the mask’s operation given the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque. For the interest of continued prosecution and the purposes of mapping the claims, the examiner understands the limitation to be “wherein the mask makes the pixel group classified as the tissue region as opaque, the tissue region including the luminal region and the extraluminal region”. Appropriate corrections required.
Claim 25, Line 1-5, recites “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region”. The Office finds the term “wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque” to render the claim indefinite. The Office understands the mask as classifying regions and the mask as being opaque. However, it is unclear what “the pixel group” is being referenced for the mask’s operation given the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque. For the interest of continued prosecution and the purposes of mapping the claims, the examiner understands the limitation to be “wherein the mask makes the pixel group classified as the tissue region as opaque, the tissue region including the luminal region and the extraluminal region”. Appropriate corrections required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 9-12, 14-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over BUCKLER et al. (US 20210390689 A1), hereinafter referenced as BUCKLER in view of LI et al. (US 20200226422 A1), hereinafter referenced as LI and in further view of ROLLINS et al. (US 20120075638 A1), hereinafter referenced as ROLLINS.
Regarding claim 1, BUCKLER explicitly teaches an information processing method (Fig. 1. Paragraph [0135]-BUCKLER discloses the systems and methods of the present disclosure utilize a hierarchical analytics framework that identifies and quantify biological properties/analytes from imaging data and then identifies and characterizes one or more pathologies based on the quantified biological properties/analytes. In paragraph [0150]-BUCKLER discloses there are three basic functionalities which may be provided by the system 100 as represented by the trainer module 110, the analyzer module 120 and the cohort tool module 130. [0150]-BUCKLER discloses The analyzer module 120 may operate independent of ground truth or validation references by implementing one or more pre-trained, e.g., machine learned algorithms for drawing its inferences. Please also see Fig. 4 and read paragraph [00]) for causing a computer to execute processing, the processing comprising:
acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body (Fig. 1. Paragraph [0150]-BUCKLER discloses the analyzer module 120 implements a hierarchical analytics framework which first identifies and quantifies biological properties/analytes 130 utilizing a combination of (i) imaging features 122 from one or more acquired images 121A of a patient 50 and (ii) non-imaging input data 121B for a patient 50 and then identifies and characterizes one or more pathologies (e.g., prognostic phenotypes) 124 based on the quantified biological properties/analytes 123. In paragraph [0152]-BUCKLER discloses the image features 122 and non-imaging inputs may be utilized by the analyzer module 120 to calculate the biological properties/analytes 123. The biological properties/analytes are typically quantitative, objective properties that may represent e.g., a presence and degree of a marker or other measurements such as structure, size, or anatomic characteristics of region of interest) are classified into a plurality of regions (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. Please also read paragraph [0185] (wherein the training of a machine learning model may include classifying, labeling and/or annotating lumen and surrounding tissue/structure), the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region (Fig. 4. Paragraph [0169]-BUCKLER discloses an initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Lumen segmentation may utilize MR imaging or CT imaging (such as use of registered pre-contrast, post-contrast CT and 2D Gaussian distributions) to define a vessel-ness function vessel segmentation may entail outer wall segmentation. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) to allow for median thickness. Ferret diameters may be employed for vessel characterization. Wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation. In paragraph [0171]-BUCKLER discloses within a cross-section through the vessel, the wall is defined by two boundaries, the inner boundary with the lumen and the outer boundary of the vessel wall, creating a donut shape in cross section. Within the donut shaped wall region, there are a discrete number of blobs);
extracting the luminal region from the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0185]);
generating edge data by applying an edge extraction filter to a classification extraction image generated based on the extracted luminal region (Fig. 4. Paragraph [0169]-BUCKLER discloses systems relating to evaluating the vascular system may advantageously include/employ algorithms for evaluating vascular structure. The systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. An initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Vessel segmentation may further entail outer wall segmentation (e.g., utilizing a minimum curvature (k2) flow to account for lumen irregularities). An edge potential map is calculated as outward-downward gradients in both contrast and non-contrast. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) in a speed function to allow for median thickness in the absence of any other edge information. Ferret diameters may be employed for vessel characterization. In further embodiments, wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. Please also read paragraph [0174-0176]);
BUCKLER fails to explicitly teach generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, based on the classification data.
However, LI explicitly teaches generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). In paragraph [0235]-LI discloses the representation of the blood vessel or the underlying tissue characterized image data obtained with regard to the blood vessel is transformed into a tissue map. In one embodiment, various colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also read paragraph [0013 and 0104]); and
generating region contour data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region (wherein initial lumen detection may be performed by a first neural network as described in US 9138147 B2 (incorporated by reference), which includes, for example, gradient filter such as sobel edge detection, selecting the longest contour of segments that exceed a predetermined angular/radial/elucidean distance threshold, calculating a weight based on the thickness and gap of scan lines corresponding to lumen tissue regions, and assessing whether the weight exceeds a predetermined discontinuity threshold). In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). Please also read paragraph [0013 and 0104]) in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region (Fig. 1. Paragraph [0073]-LI discloses FIGS. 16-18 show various tissue map representations generated using an OCT imaging pullback of an artery with various indicia integrated into a user interface displaying the various tissue maps to support diagnosis and treatment plans (wherein FIG. 15, for example, shows a schematic representation of various cut planes/rings/boundaries R1 to Rn that are shown along a 3D artery representation 1425 extending in the proximal and distal direction). In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425 (wherein outputted results may include masking, detected boundaries of a lumen, the thickness of luminal layers, graphical elements describing thickness such as hatching and a graphical user interface where regions can be selected, filtered and modified by class or type). Therefore, it would have been obvious to a person of ordinary skill in the art to remove a portion from an inner surface of the living tissue region facing the luminal region. LI teaches semantic segmentation, thickness and proximity measurements as well as the detection and display of individual luminal regions using thickness/distance ranges and thresholds. Moreover, LI teaches filtering, removing and reconstructing areas within the luminal region and generating proximal and distal views where a portion of living tissue facing the luminal region has been removed. Thus, this would have improved the visualizations and semantic segmentations of luminal regions as well as allowed for greater isolation of areas. Please also see US 9138147 B2 “Lumen morphology image reconstruction based on the scan line data of OCT” which is incorporated by reference in its entirety, see Fig. 2, 3C, 5, and 8 and read paragraph [0135, 0142 and 0202]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, based on the classification data.
Wherein BUCKLER’s method having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, based on the classification data.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
BUCKLER in view of LI fail to explicitly teach generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range.
However, ROLLINS explicitly teaches generating thick-line edge data (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask, and the classification results for regions such as plaques may be displayed using different colors). Please also read paragraph [0097]) by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range (Fig. 2. Paragraph [0054]-ROLLINS discloses the entire contour is stopped when there is little change of the texture difference inside and outside the contour within a predefined layer having a width of w where w is chosen to safely cover the depth of all calcified plaques. w is set to 2 mm, corresponding to the contour depth limit of an example OCT system (wherein a texture-based active contour method, such as a Gabor filter may be implemented to magnify and find the boundary of the adventitial tissue as the vessel wall, a morphological opening may be used to separate the arterial wall and further classification behind the contour may be performed). Further in paragraph [0067]-ROLLINGS discloses the CP localization step 51 locates the calcified lesion at a coarse level. Once the binary edge areas within a 40 degree segment of the segmented vessel wall 46 or 47 as shown in FIGS. 8, 9A and 9B, exceeds a threshold of 0.02 mm.sup.2, the segment is marked as initial ROI. The segment length is increased until there is no change in edge intensity and stored as the final ROI for the CP. Please also see Fig. 4, 8-9 and 12, and read paragraph [0084-0087]);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of ROLLINS of having generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range.
Wherein BUCKLER’s method having generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
BUCKLER in view of LI fail to explicitly teach generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range.
However, ROLLINS explicitly teaches generating thick-line edge data (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask, and the classification results for regions such as plaques may be displayed using different colors). Please also read paragraph [0097]) by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness with a predetermined range (Fig. 2. Paragraph [0054]-ROLLINS discloses the entire contour is stopped when there is little change of the texture difference inside and outside the contour within a predefined layer having a width of w where w is chosen to safely cover the depth of all calcified plaques. w is set to 2 mm, corresponding to the contour depth limit of an example OCT system (wherein a texture-based active contour method, such as a Gabor filter may be implemented to magnify and find the boundary of the adventitial tissue as the vessel wall, a morphological opening may be used to separate the arterial wall and further classification behind the contour may be performed). Further in paragraph [0067]-ROLLINGS discloses the CP localization step 51 locates the calcified lesion at a coarse level. Once the binary edge areas within a 40 degree segment of the segmented vessel wall 46 or 47 as shown in FIGS. 8, 9A and 9B, exceeds a threshold of 0.02 mm.sup.2, the segment is marked as initial ROI. The segment length is increased until there is no change in edge intensity and stored as the final ROI for the CP. Please also see Fig. 4, 8-9 and 12, and read paragraph [0084-0087]);
Regarding claim 3, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing method according to claim 1, BUCKLER further teaches further comprising:
acquiring, as the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0185]), three-dimensional classification data for the three-dimensional biomedical image data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images);
generating, based on the three-dimensional classification data, thick boundary surface data in which a boundary surface between the living tissue region and the luminal region has a thickness within a predetermined range (Fig. 5. Paragraph [0074]-BUCKLER discloses a normalized radial distance may have a value of 0 at an inner (inner wall luminal boundary) and value of 1 at an outer boundary (outer wall boundary). In paragraph [0186]-BUCKLER discloses the radial distance may be defined based on the shortest distance to the inner luminal surface and the shortest distance to the outer adventitial surface. The relative radial distance is computed as r(x)=L(x)/(L(x)−V(x)). It has a value of 0 at the luminal surface and 1 at the adventitial surface. In paragraph [0188]-BUCKLER discloses another wall coordinate that is used is normalized wall thickness. The absolute wall thickness is easily calculated as w.sub.abs(x)=L(x)−V(x). In order to normalize it to the range of [0-1], one may determine that maximum possible wall thickness when the lumen approaches zero size and is completely eccentric and near the outer surface. In this case the maximum diameter is the maximum Feret diameter of the vessel, D.sub.max. Thus, the relative wall thickness is computed as w(x)=w.sub.abs(x)/D.sub.max. Please also read paragraph [0169 and 0175]);
BUCKLER fails to explicitly teach generating, based on the three-dimensional classification data, a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data.
However, LI explicitly teaches generating, based on the three-dimensional classification data (Fig. 12A-B. Paragraph [0114]-LI discloses various features, regions, types, and/or classes of tissue and regions, pixels, contours and boundaries in images may be tagged or identified relative to image data to obtain annotated image data such as ground truth masks. In paragraph [0118]-LI discloses Step 100. Initially, a set of ground truth data, such as a ground truth masks is established by reviewing and annotating the set of image data. Step 102. Pixels corresponding to a ground truth annotation may be stored in persistent electronic memory such as the database of FIG. 1. In paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105). In paragraph [0133]-LI discloses the neural network architecture may be 2D or 3D network and as such, operable to process 2D data and 3D data), a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data (Fig. 12A-B. Paragraph [0104]-LI discloses each feature or class identified, such as ADV, EEL, IEL, L, P, I, Q may be generated as a mask or a predictive mask using one or more of the trained neural networks (wherein ADV, EEL, IEL, L, P, I, Q corresponds to lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M,). Indicia corresponding to the output results can be show using color coded indicia and other indicia. In paragraph [0121]-LI discloses generating K probability masks for each of K classes/types. Step 106. Please also see Fig. 3C and 10A-B); and
applying the mask (Fig. 2A-H. Paragraph [0122]-LI discloses the method may include generating final predictive output for each frame of input image data. Step 107. Each of the K probability maps for reach of the different K classes/types, are compared on a per pixel basis and assessed such that each pixel for a given image frame is assessed and then a final predictive result is assigned to each pixel. The final predictive results are predictive output masks that include one or more indicia corresponding to a type/class. The method includes displaying final predictive output images from neural network/machine learning system with class/type indicia. Step 108) to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data (Fig. 2A-H. Paragraph [0134]-LI discloses color code indicia have been used with blue for lumen, red for calcium, and green for media. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425. In paragraph [0235]-LI discloses colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also see Fig. 5 and 15-18, and read paragraph [0135-0136, 0194-0195, 0209, 0214, 0230]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having generating, based on the three-dimensional classification data, a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data.
Wherein BUCKLER’s method having generating, based on the three-dimensional classification data, a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 4, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing method according to claim 3, BUCKLER fails to explicitly teach further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range to three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
However, LI explicitly teaches further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range (Fig. 2A-H. Paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425. In paragraph [0235]-LI discloses colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also see Fig. 3C, 5 and 15-18, and read paragraph [0134-0136, 0194-0195, 0209, 0214, 0230]) to three-dimensional edge data (Fig. 12A-12B. Paragraph [0133]-LI discloses the neural network architecture may be 2D or 3D network and as such, operable to process 2D data and 3D data. Further in paragraph [0135]-LI discloses the carpet view or lines projections 192 are used to create a tissue map 198. The carpet view includes 3D data. Please also read paragraph [0142]) generated by applying an edge extraction filter to classification image data (Fig. 2A-H. Paragraph [0135]-LI discloses a binary morphological reconstruct filter is applied to media and calcium carpet view image 198. Please also read paragraph [0236]) generated based on the three-dimensional classification data (Fig. 2A-H. Paragraph [0134]-LI discloses after media and calcium detection process, each frame in polar space will have corresponding frames/masks for media M and calcium Ca regions/features of interest. These frames/masks may include lumen L and other classes that were used to train neural network for ROI/FOI detection. A set of four output image masks/frames 190 is shown. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. The carpet view or lines projections 192 are used to create a tissue map 198).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
Wherein BUCKLER’s method having three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 5, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing method according to claim 4, BUCKLER fails to explicitly teach further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
However, LI explicitly teaches further comprising: generating the thick boundary surface data (Fig. 2A-H. Paragraph [0134]-LI discloses after media and calcium detection process, each frame in polar space will have corresponding frames/masks for media M and calcium Ca regions/features of interest. These frames/masks may include lumen L and other classes that were used to train neural network for ROI/FOI detection. A set of four output image masks/frames 190 is shown. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. The carpet view or lines projections 192 are used to create a tissue map 198) by applying a three-dimensional expansion filter to the edge data (Fig. 2A. Paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. A binary morphological reconstruct filter is applied to media and calcium carpet view image 198 (wherein a binary morphological reconstruct filter includes opening, closing, dilation and erosion). Please also read paragraph [0220 and 0229]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
Wherein BUCKLER’s method having further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 9, BUCKLER in view of LI and in further view of ROLLINS explicitly teaches the information processing method according to claim 1, BUCKLER further teaches wherein the biomedical image data (Fig. 4. Paragraph [0150]-BUCKLER discloses there are three basic functionalities which may be provided by the system 100 as represented by the trainer module 110, the analyzer module 120 and the cohort tool module 130. The analyzer module 120 advantageously implements a hierarchical analytics framework which first identifies and quantifies biological properties/analytes 130 utilizing a combination of (i) imaging features 122 from one or more acquired images 121A of a patient 50 and (ii) non-imaging input data 121B for a patient 50 and then identifies and characterizes one or more pathologies (e.g., prognostic phenotypes) 124 based on the quantified biological properties/analytes 123) is structured (Fig. 4. Paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. Please also see Fig. 6, and 8-10 and read paragraph [0169 and 0185]) from tomographic image data of a living body acquired utilizing an image acquisition catheter (Fig. 4. Claim 17-BUCKLER discloses wherein the radiological dataset includes computed tomography (CT), spectral computed tomography (spectral CT), computed tomography angiography (CTA), cardiac computed tomography angiography (CCTA), intra-vascular ultrasound (IVUS) (wherein IVUS is an image acquisition catheter). Please also see claim 1);
and wherein the classification data is data in which the luminal region is further classified into a first luminal region into which the image acquisition catheter is inserted and a second luminal region into which the image acquisition catheter is not inserted (Fig. 4. Paragraph [0191]-BUCKLER discloses FIG. 9 illustrates some complex vessel topologies which can be accounted for using the techniques. If a segmented view (cross-sectional slice) includes more than one lumen, one can account for this by performing a watershed transform on r in order to split up wall into domains belonging to each lumen after which each domain may be separately considered/analyzed. Therefore, it would have been obvious to a person of ordinary skill to insert a catheter into one luminal region and not another. Buckler classifies multiple different luminal regions within one or more lumens that may be within close proximity to each other. In addition, BUCKLER uses localized IVUS data from a lumen in combination with other non-invasive imaging methods such as CT, which are not limited by physical boundaries and typically scan a larger area. As a result, these other non-invasive imaging methods typically provide information that can work in conjunction with localized IVUS data to classify areas beyond the single luminal region. Thus, it would be obvious to classify two luminal regions while only acquiring IVUS data from one of these regions. This would improve the ability to acquire diagnostic data and decrease the level of invasiveness).
Regarding claim 10, BUCKLER in view of LI and in further view of ROLLINS explicitly teaches the information processing method according to claim 9, BUCKLER further teaches further comprising:
receiving from the first luminal region and the second luminal region, selection of one or more selected regions (Fig. 4. Paragraph [0191]-BUCKLER discloses FIG. 9 illustrates some complex vessel topologies which can be accounted for using the techniques. If a segmented view (cross-sectional slice) includes more than one lumen, one can account for this by performing a watershed transform on r in order to split up wall into domains belonging to each lumen after which each domain may be separately considered/analyzed. Please also read paragraph [0169, 0170-0171 and 0175); and
generating thick-line edge data having a thickness within a predetermined range at the boundary between the living tissue region and the luminal region, or thick boundary surface data having a thickness within a predetermined range based on the three-dimensional classification data for the three-dimensional biomedical image data acquired as the classification data, based on a boundary or a boundary surface between the living tissue region and the selected region (Fig. 5. Paragraph [0074]-BUCKLER discloses a normalized radial distance may have a value of 0 at an inner (inner wall luminal boundary) and value of 1 at an outer boundary (outer wall boundary). In paragraph [0186]-BUCKLER discloses the radial distance may be defined based on the shortest distance to the inner luminal surface and the shortest distance to the outer adventitial surface. The relative radial distance is computed as r(x)=L(x)/(L(x)−V(x)). It has a value of 0 at the luminal surface and 1 at the adventitial surface. In paragraph [0188]-BUCKLER discloses another wall coordinate that is used is normalized wall thickness. The absolute wall thickness is easily calculated as w.sub.abs(x)=L(x)−V(x). In order to normalize it to the range of [0-1], one may determine that maximum possible wall thickness when the lumen approaches zero size and is completely eccentric and near the outer surface. In this case the maximum diameter is the maximum Feret diameter of the vessel, D.sub.max. Thus, the relative wall thickness is computed as w(x)=w.sub.abs(x)/D.sub.max. Please also read paragraph [0169 and 0175]).
Regarding claim 11, BUCKLER in view of LI and in further view of ROLLINS explicitly teaches the information processing method according to claim 1, BUCKLER further teaches wherein the region contour data includes a plurality of pieces of two-dimensional region contour data generated respectively based on the plurality of pieces of two-dimensional classification data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. Please also read paragraph [0169 and 0175]); and
generating based on the two-dimensional region contour data, a three-dimensional image (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images. Please also read paragraph [0169).
BUCKLER is silent on wherein the biomedical image data includes a plurality of pieces of the tomographic image data generated in time series; wherein the classified data includes a plurality of pieces of two-dimensional classification data generated respectively based on the plurality of pieces of the tomographic image data.
However, LI explicitly teaches wherein the biomedical image data includes a plurality of pieces of the tomographic image data (Fig. 1. Paragraph [0080]-LI discloses the data collection system 3 includes a noninvasive imaging system such as computer aided tomography, or other suitable noninvasive imaging technology. Angiography system 20 is configured to noninvasively image the subject 4 such that frames of angiography data, typically in the form of frames of image data, are generated while a pullback procedure is performed using a probe 30 such that a blood vessel in region 25 of subject 4 is imaged using angiography in one or more imaging technologies such as OCT or IVUS) generated in time series (Fig. 1. Paragraph [0086]-LI discloses the data collection system 40 and the angiography system 20 have a shared clock or other timing signals configured to synchronize angiography video frame time stamps and OCT image frame time stamps);
wherein the classified data (Fig. 1. Paragraph [0080]-LI discloses the data collection system 3 includes a noninvasive imaging system such as computer aided tomography, or other suitable noninvasive imaging technology. Angiography system 20 is configured to noninvasively image the subject 4 such that frames of angiography data, typically in the form of frames of image data, are generated while a pullback procedure is performed using a probe 30 such that a blood vessel in region 25 of subject 4 is imaged using angiography in one or more imaging technologies such as OCT or IVUS. In paragraph [0120]-LI discloses training a neural network of a machine learning system using set of annotated polar images is performed. Step 104. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein semantic segmentation is performed)) includes a plurality of pieces of two-dimensional classification data generated respectively based on the plurality of pieces of the tomographic image data (Fig. 1. Paragraph [0133]-LI discloses the neural network architecture may be 2D or 3D network and as such, operable to process 2D data and 3D data. In paragraph [0142]-LI discloses lumen detection is implemented using a 2D or a 3D neural network that is trained with annotated images such as ground truth masks with the lumen boundary identified. Please also read paragraph [0134-0136]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having wherein the biomedical image data includes a plurality of pieces of the tomographic image data generated in time series; wherein the classified data includes a plurality of pieces of two-dimensional classification data generated respectively based on the plurality of pieces of the tomographic image data.
Wherein BUCKLER’s method having wherein the biomedical image data includes a plurality of pieces of the tomographic image data generated in time series; wherein the classified data includes a plurality of pieces of two-dimensional classification data generated respectively based on the plurality of pieces of the tomographic image data.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 12, BUCKLER in view of LI and in further view of ROLLINS explicitly teaches the information processing method according to claim 1, BUCKLER further teaches the classification data includes three-dimensional classification data for the three-dimensional biomedical image data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images. Please also read paragraph [0169]);
the region contour data includes three-dimensional region contour data generated based on the three-dimensional classification data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images. Please also read paragraph [0169]); and
generating a three-dimensional image based on the three-dimensional region contour data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images. Please also read paragraph [0169]).
BUCKLER fails to explicitly teach wherein the biomedical image data is three-dimensional biomedical image data structured from a plurality of pieces of the tomographic image data generated in time series.
However, LI explicitly teaches wherein the biomedical image data is three-dimensional biomedical image data structured from a plurality of pieces of the tomographic image data (Fig. 1. Paragraph [0080]-LI discloses the data collection system 3 includes a noninvasive imaging system such as computer aided tomography, or other suitable noninvasive imaging technology. Angiography system 20 is configured to noninvasively image the subject 4 such that frames of angiography data, typically in the form of frames of image data, are generated while a pullback procedure is performed using a probe 30 such that a blood vessel in region 25 of subject 4 is imaged using angiography in one or more imaging technologies such as OCT or IVUS) generated in time series generated in time series (Fig. 1. Paragraph [0086]-LI discloses the data collection system 40 and the angiography system 20 have a shared clock or other timing signals configured to synchronize angiography video frame time stamps and OCT image frame time stamps);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI of having an information processing method for causing a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions, the plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having wherein the biomedical image data is three-dimensional biomedical image data structured from a plurality of pieces of the tomographic image data generated in time series.
Wherein BUCKLER’s method having wherein the biomedical image data is three-dimensional biomedical image data structured from a plurality of pieces of the tomographic image data generated in time series.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 14, BUCKLER explicitly teaches an information processing apparatus (Fig. 1, #100 called a system. Paragraph [0150]), comprising:
a classification data acquisition unit configured to acquire classification data (Fig. 1. Paragraph [0150]-BUCKLER discloses the analyzer module 120 implements a hierarchical analytics framework which first identifies and quantifies biological properties/analytes 130 utilizing a combination of (i) imaging features 122 from one or more acquired images 121A of a patient 50 and (ii) non-imaging input data 121B for a patient 50 and then identifies and characterizes one or more pathologies (e.g., prognostic phenotypes) 124 based on the quantified biological properties/analytes 123. Please also see Fig. 4 and read paragraph [00]) in which pixels constituting biomedical image data indicating an internal structure of a living body (Fig. 1. Paragraph [0152]-BUCKLER discloses the image features 122 and non-imaging inputs may be utilized by the analyzer module 120 to calculate the biological properties/analytes 123. The biological properties/analytes are typically quantitative, objective properties that may represent e.g., a presence and degree of a marker or other measurements such as structure, size, or anatomic characteristics of region of interest) are classified into a plurality of regions including a living tissue region in which a luminal region exists (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. Please also read paragraph [0185] (wherein the training of a machine learning model may include classifying, labeling and/or annotating lumen and surrounding tissue/structure), the luminal region, and an extraluminal region outside the living tissue region (Fig. 1. Paragraph [0169]-BUCKLER discloses an initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Lumen segmentation may utilize MR imaging or CT imaging (such as use of registered pre-contrast, post-contrast CT and 2D Gaussian distributions) to define a vessel-ness function vessel segmentation may entail outer wall segmentation. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) to allow for median thickness. Ferret diameters may be employed for vessel characterization. Wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation. In paragraph [0171]-BUCKLER discloses within a cross-section through the vessel, the wall is defined by two boundaries, the inner boundary with the lumen and the outer boundary of the vessel wall, creating a donut shape in cross section. Within the donut shaped wall region, there are a discrete number of blobs); and
a generation unit configured to:
extract the luminal region from the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0174-0176 and 0185]);
generate edge data by applying an edge extraction filter to a classification extraction image generated based on the extracted luminal region (Fig. 4. Paragraph [0169]-BUCKLER discloses systems relating to evaluating the vascular system may advantageously include/employ algorithms for evaluating vascular structure. The systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. An initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Vessel segmentation may further entail outer wall segmentation (e.g., utilizing a minimum curvature (k2) flow to account for lumen irregularities). An edge potential map is calculated as outward-downward gradients in both contrast and non-contrast. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) in a speed function to allow for median thickness in the absence of any other edge information. Ferret diameters may be employed for vessel characterization. In further embodiments, wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. Please also read paragraph [0174-0176]);
BUCKLER fails to explicitly teach generate, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generate region contour data in which a portion where a thickness exceeds from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
However, LI explicitly teaches generate, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). In paragraph [0235]-LI discloses the representation of the blood vessel or the underlying tissue characterized image data obtained with regard to the blood vessel is transformed into a tissue map. In one embodiment, various colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also read paragraph [0013 and 0104]); and
generate region contour data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region (wherein initial lumen detection may be performed by a first neural network as described in US 9138147 B2 (incorporated by reference), which includes, for example, gradient filter such as sobel edge detection, selecting the longest contour of segments that exceed a predetermined angular/radial/elucidean distance threshold, calculating a weight based on the thickness and gap of scan lines corresponding to lumen tissue regions, and assessing whether the weight exceeds a predetermined discontinuity threshold). In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). Please also read paragraph [0013 and 0104]) in which a portion where a thickness exceeds from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data (Fig. 1. Paragraph [0073]-LI discloses FIGS. 16-18 show various tissue map representations generated using an OCT imaging pullback of an artery with various indicia integrated into a user interface displaying the various tissue maps to support diagnosis and treatment plans (wherein FIG. 15, for example, shows a schematic representation of various cut planes/rings/boundaries R1 to Rn that are shown along a 3D artery representation 1425 extending in the proximal and distal direction). In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425 (wherein outputted results may include masking, detected boundaries of a lumen, the thickness of luminal layers, graphical elements describing thickness such as hatching and a graphical user interface where regions can be selected, filtered and modified by class or type). Therefore, it would have been obvious to a person of ordinary skill in the art to remove a portion from an inner surface of the living tissue region facing the luminal region. LI teaches semantic segmentation, thickness and proximity measurements as well as the detection and display of individual luminal regions using thickness/distance ranges and thresholds. Moreover, LI teaches filtering, removing and reconstructing areas within the luminal region and generating proximal and distal views where a portion of living tissue on the inner surface facing the luminal region has been removed. Thus, this would have improved the visualizations and semantic segmentations of luminal regions as well as allowed for greater isolation of areas. Please also see US 9138147 B2 “Lumen morphology image reconstruction based on the scan line data of OCT” which is incorporated by reference in its entirety, see Fig. 2, 3C, 5, and 8 and read paragraph [0135, 0142 and 0202]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER of having an information processing apparatus, comprising: a classification data acquisition unit configured to acquire classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having generate, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generate region contour data in which a portion where a thickness exceeds from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
Wherein BUCKLER’s information processing apparatus having generate, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generate region contour data in which a portion where a thickness exceeds from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
The motivation behind the modification would have been to obtain an information processing apparatus that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
BUCKLER in view of LI fail to explicitly teach generate thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range.
However, ROLLINS explicitly teaches generate thick-line edge data by applying an expansion filter to the edge data (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask, and the classification results for regions such as plaques may be displayed using different colors). Please also read paragraph [0054-0055, 0065-0068 and 0097]) such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range (Fig. 2. Paragraph [0054]-ROLLINS discloses the entire contour is stopped when there is little change of the texture difference inside and outside the contour within a predefined layer having a width of w where w is chosen to safely cover the depth of all calcified plaques. w is set to 2 mm, corresponding to the contour depth limit of an example OCT system (wherein a texture-based active contour method, such as a Gabor filter may be implemented to magnify and find the boundary of the adventitial tissue as the vessel wall, a morphological opening may be used to separate the arterial wall and further classification behind the contour may be performed). Further in paragraph [0067]-ROLLINGS discloses the CP localization step 51 locates the calcified lesion at a coarse level. Once the binary edge areas within a 40 degree segment of the segmented vessel wall 46 or 47 as shown in FIGS. 8, 9A and 9B, exceeds a threshold of 0.02 mm.sup.2, the segment is marked as initial ROI. The segment length is increased until there is no change in edge intensity and stored as the final ROI for the CP. Please also see Fig. 4, 8-9 and 12, and read paragraph [0084-0087]);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI of having an information processing apparatus, comprising: a classification data acquisition unit configured to acquire classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of ROLLINS of having generate thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range;
Wherein BUCKLER’s information processing apparatus having generate thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range.
The motivation behind the modification would have been to obtain an information processing apparatus that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
Regarding claim 15, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing apparatus according to claim 14, BUCKLER further teaches further comprising:
an output unit configured to output a three-dimensional image generated based on the region contour data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images).
Regarding claim 16, BUCKLER explicitly teaches a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing (Fig. 1. Paragraph [0150]-BUCKLER discloses with initial reference to FIG. 1, a schematic of an exemplary system 100 is depicted. In paragraph [0399]-BUCKLER discloses systems and methods may be implemented in digital electronic circuitry, in computer hardware, firmware, and/or software. The implementation can be as a computer program product (i.e., a computer program tangibly embodied in an information carrier). The implementation can, for example, be in a machine-readable storage device and/or in a propagated signal, for execution by, or to control the operation of, data processing apparatus. The implementation can, for example, be a programmable processor, a computer, and/or multiple computers), the processing comprising:
acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body (Fig. 1. Paragraph [0150]-BUCKLER discloses the analyzer module 120 implements a hierarchical analytics framework which first identifies and quantifies biological properties/analytes 130 utilizing a combination of (i) imaging features 122 from one or more acquired images 121A of a patient 50 and (ii) non-imaging input data 121B for a patient 50 and then identifies and characterizes one or more pathologies (e.g., prognostic phenotypes) 124 based on the quantified biological properties/analytes 123. In paragraph [0152]-BUCKLER discloses the image features 122 and non-imaging inputs may be utilized by the analyzer module 120 to calculate the biological properties/analytes 123. The biological properties/analytes are typically quantitative, objective properties that may represent e.g., a presence and degree of a marker or other measurements such as structure, size, or anatomic characteristics of region of interest) are classified into a plurality of regions including a living tissue region in which a luminal region exists (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. Please also read paragraph [0185]), the luminal region, and an extraluminal region outside the living tissue region (Fig. 4. Paragraph [0169]-BUCKLER discloses an initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Lumen segmentation may utilize MR imaging or CT imaging (such as use of registered pre-contrast, post-contrast CT and 2D Gaussian distributions) to define a vessel-ness function vessel segmentation may entail outer wall segmentation. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) to allow for median thickness. Ferret diameters may be employed for vessel characterization. Wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation. In paragraph [0171]-BUCKLER discloses within a cross-section through the vessel, the wall is defined by two boundaries, the inner boundary with the lumen and the outer boundary of the vessel wall, creating a donut shape in cross section. Within the donut shaped wall region, there are a discrete number of blobs); and
extracting the luminal region from the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0174-0176 and 0185]);
generating edge data by applying an edge extraction filter to a classification extraction image generated based on the extracted luminal region (Fig. 4. Paragraph [0169]-BUCKLER discloses systems relating to evaluating the vascular system may advantageously include/employ algorithms for evaluating vascular structure. The systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. An initial lumen segmentation may utilize a confidence connected filter (e.g., carotid, vertebral, femoral, etc.) to distinguish the lumen. Vessel segmentation may further entail outer wall segmentation (e.g., utilizing a minimum curvature (k2) flow to account for lumen irregularities). An edge potential map is calculated as outward-downward gradients in both contrast and non-contrast. Outer wall segmentation may utilize cumulative distribution functions (incorporating prior distributions of wall thickness, e.g., from 1-2 adjoining levels) in a speed function to allow for median thickness in the absence of any other edge information. Ferret diameters may be employed for vessel characterization. In further embodiments, wall thickness may be calculated as the sum of the distance to lumen plus the distance to the outer wall. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. Please also read paragraph [0174-0176]);
BUCKLER fails to explicitly teach generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
However, LI explicitly teaches generate, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). In paragraph [0235]-LI discloses the representation of the blood vessel or the underlying tissue characterized image data obtained with regard to the blood vessel is transformed into a tissue map. In one embodiment, various colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also read paragraph [0013 and 0104]); and
generating region contour data (Fig. 12A-B. Paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region (wherein initial lumen detection may be performed by a first neural network as described in US 9138147 B2 (incorporated by reference), which includes, for example, gradient filter such as sobel edge detection, selecting the longest contour of segments that exceed a predetermined angular/radial/elucidean distance threshold, calculating a weight based on the thickness and gap of scan lines corresponding to lumen tissue regions, and assessing whether the weight exceeds a predetermined discontinuity threshold). In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105. Generating K probability masks for each of K classes/types. Step 106 (wherein the features, regions, types, and/or classes include one or more lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, stent, calcium, radio opaque marker, fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, length, and thickness). Please also read paragraph [0013 and 0104]) in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data (Fig. 1. Paragraph [0073]-LI discloses FIGS. 16-18 show various tissue map representations generated using an OCT imaging pullback of an artery with various indicia integrated into a user interface displaying the various tissue maps to support diagnosis and treatment plans (wherein FIG. 15, for example, shows a schematic representation of various cut planes/rings/boundaries R1 to Rn that are shown along a 3D artery representation 1425 extending in the proximal and distal direction). In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425 (wherein outputted results may include masking, detected boundaries of a lumen, the thickness of luminal layers, graphical elements describing thickness such as hatching and a graphical user interface where regions can be selected, filtered and modified by class or type). Therefore, it would have been obvious to a person of ordinary skill in the art to apply a mask to remove a portion of living tissue region where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold. LI teaches edge detection, semantic segmentation, thickness and proximity measurements as well as the detection, ablation and display of individual luminal regions using thickness/distance ranges, weights and thresholds. Moreover, LI teaches masking, filtering, removing and reconstructing regions/layers within the luminal region and generating proximal and distal views where a portion of living tissue on the inner surface facing the luminal region has been removed. Thus, this would have improved the visualizations and semantic segmentations of luminal regions as well as allowed for greater isolation of areas. Please also see US 9138147 B2 “Lumen morphology image reconstruction based on the scan line data of OCT” which is incorporated by reference in its entirety, see Fig. 2, 3C, 5, and 8 and read paragraph [0135, 0142 and 0202]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER of having a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
Wherein BUCKLER’s non-transitory computer-readable medium having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and generating region contour data in which a portion where a thickness from an inner surface of the living tissue region facing the luminal region exceeds a predetermined threshold is removed from the living tissue region, by applying the mask to the thick-line edge data.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
BUCKLER in view of LI fail to explicitly teach generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range.
However, ROLLINS explicitly teaches generating thick-line edge data by applying an expansion filter to the edge data (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask, and the classification results for regions such as plaques may be displayed using different colors). Please also read paragraph [0097]) such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range (Fig. 2. Paragraph [0054]-ROLLINS discloses the entire contour is stopped when there is little change of the texture difference inside and outside the contour within a predefined layer having a width of w where w is chosen to safely cover the depth of all calcified plaques. w is set to 2 mm, corresponding to the contour depth limit of an example OCT system (wherein a texture-based active contour method, such as a Gabor filter may be implemented to magnify and find the boundary of the adventitial tissue as the vessel wall, a morphological opening may be used to separate the arterial wall and further classification behind the contour may be performed). Further in paragraph [0067]-ROLLINGS discloses the CP localization step 51 locates the calcified lesion at a coarse level. Once the binary edge areas within a 40 degree segment of the segmented vessel wall 46 or 47 as shown in FIGS. 8, 9A and 9B, exceeds a threshold of 0.02 mm.sup.2, the segment is marked as initial ROI. The segment length is increased until there is no change in edge intensity and stored as the final ROI for the CP. Please also see Fig. 4, 8-9 and 12, and read paragraph [0084-0087]);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI of having a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of ROLLINS of having generating thick-line edge data by applying an expansion filter to the edge data such that a boundary between the living tissue region and the luminal region has a thickness within a predetermined range.
Wherein BUCKLER’s non-transitory computer-readable medium having generating region contour data in which a portion where a thickness exceeds a predetermined threshold is removed from the living tissue region, based on the classification data.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
Regarding claim 18, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the non-transitory computer-readable medium according to claim 16, BUCKLER further teaches further comprising: acquiring, as the classification data (Fig. 4. Paragraph [0169]-BUCKLER discloses the systems may employ, e.g., a target/vessel segment/cross-section model for segmenting the underlying structure of an imaged vessel. Lumen and/or wall segmentations may be done using semantic segmentation using, for example, CNNs. In paragraph [0171]-BUCKLER discloses introduced is a novel model for classification of composition of vascular plaque components. The multi-scale model computes the statistics of each contiguous region of a given analyte type, which may be referred to as a ‘blob’. Each blob is assigned a label of analyte type and various shape descriptors are computed. In paragraph [0175]-BUCKLER discloses the multi-scale vessel wall analyte map may advantageously include wall-level segmentation 410 (e.g., a cross-sectional slice of the vessel), blob-level segmentation and pixel-level segmentation 430 (e.g., based on individual image pixels. E.g., A=(B,C) may be defined as a map of vessel wall class labels, wherein B is a set of blobs (cross-sectionally contiguous regions of non-background wall sharing a label) and C is a set of blob couples or pairs. A(x)=a may be defined as the class label of pixel x where a∈{‘CALC’, ‘LRNC’, ‘FIBR’, ‘IPH’, ‘background’ } (compositional characteristics). Please also read paragraph [0185]), three-dimensional classification data for the three-dimensional biomedical image data (Fig. 4. Paragraph [0161]-BUCKLER discloses a sample patient report 300 is depicted in FIG. 3. The sample patient report may further include visualizations 340, e.g., 2D and/or 3D visualizations of imaging data. In paragraph [0170]-BUCKLER discloses we define an analyte blob to be a spatially contiguous region, in 2D, 3D, or 4D images, of one class of biological analyte. In paragraph [0175]-BUCKLER discloses the model is used to classify wall composition in 3D radiological images);
generating, based on the three-dimensional classification data, thick boundary surface data in which a boundary surface between the living tissue region and the luminal region has a thickness within a predetermined range (Fig. 5. Paragraph [0074]-BUCKLER discloses a normalized radial distance may have a value of 0 at an inner (inner wall luminal boundary) and value of 1 at an outer boundary (outer wall boundary). In paragraph [0186]-BUCKLER discloses the radial distance may be defined based on the shortest distance to the inner luminal surface and the shortest distance to the outer adventitial surface. The relative radial distance is computed as r(x)=L(x)/(L(x)−V(x)). It has a value of 0 at the luminal surface and 1 at the adventitial surface. In paragraph [0188]-BUCKLER discloses another wall coordinate that is used is normalized wall thickness. The absolute wall thickness is easily calculated as w.sub.abs(x)=L(x)−V(x). In order to normalize it to the range of [0-1], one may determine that maximum possible wall thickness when the lumen approaches zero size and is completely eccentric and near the outer surface. In this case the maximum diameter is the maximum Feret diameter of the vessel, D.sub.max. Thus, the relative wall thickness is computed as w(x)=w.sub.abs(x)/D.sub.max. Please also read paragraph [0169 and 0175]);
BUCKLER fails to explicitly teach generating, based on the three-dimensional classification data, a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data.
However, LI explicitly teaches generating, based on the three-dimensional classification data (Fig. 12A-B. Paragraph [0114]-LI discloses various features, regions, types, and/or classes of tissue and regions, pixels, contours and boundaries in images may be tagged or identified relative to image data to obtain annotated image data such as ground truth masks. In paragraph [0118]-LI discloses Step 100. Initially, a set of ground truth data, such as a ground truth masks is established by reviewing and annotating the set of image data. Step 102. Pixels corresponding to a ground truth annotation may be stored in persistent electronic memory such as the database of FIG. 1. In paragraph [0120]-LI discloses lumen detection is performed as an initial detection step such that the ground truth masks include a lumen boundary or lumen feature or lumen region. In paragraph [0121]-LI discloses once the MLS is trained, inputting image data to the neural network is performed to generate a set of image data with predictions, detections, classifications, etc. of the various features/regions of interest. Step 105). In paragraph [0133]-LI discloses the neural network architecture may be 2D or 3D network and as such, operable to process 2D data and 3D data), a three-dimensional mask corresponding to a pixel group classified as the living tissue region in the biomedical image data (Fig. 12A-B. Paragraph [0104]-LI discloses each feature or class identified, such as ADV, EEL, IEL, L, P, I, Q may be generated as a mask or a predictive mask using one or more of the trained neural networks (wherein ADV, EEL, IEL, L, P, I, Q corresponds to lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M,). Indicia corresponding to the output results can be show using color coded indicia and other indicia. In paragraph [0121]-LI discloses generating K probability masks for each of K classes/types. Step 106. Please also see Fig. 3C and 10A-B); and
applying the mask to the thick boundary surface data and generating the three- dimensional region contour data as the region contour data (Fig. 2A-H. Paragraph [0134]-LI discloses color code indicia have been used with blue for lumen, red for calcium, and green for media. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425. In paragraph [0235]-LI discloses colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also see Fig. 5 and 15-18, and read paragraph [0135-0136, 0194-0195, 0209, 0214, 0230]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick-line edge data and the region contour data.
Wherein BUCKLER’s non-transitory computer-readable medium having generating, based on the classification data, a mask corresponding to a pixel group classified as the living tissue region in the biomedical image data; and applying the mask to the thick-line edge data and the region contour data.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 19, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the non-transitory computer-readable medium according to claim 18, BUCKLER fails to explicitly teach further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range to three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
However, LI explicitly teaches further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range (Fig. 2A-H. Paragraph [0229]-LI discloses a given tissue map may show the extent of calcium plaque. The intensity value corresponds to the thickness of calcium plaque in millimeters as shown by legend that ranges from about 0.5 mm to about 1.5 mm. Any classes or types for ROIs/FOIs detected using methods disclosed herein may be displayed using a tissue map representation such as tissue map 1425. In paragraph [0235]-LI discloses colors, shapes, hatching, masks, boundaries, and other graphical elements or overlays are used to identify or segment detected tissue types and/or regions of interest in the tissue map. Please also see Fig. 3C, 5 and 15-18, and read paragraph [0134-0136, 0194-0195, 0209, 0214, 0230]) to three-dimensional edge data (Fig. 12A-12B. Paragraph [0133]-LI discloses the neural network architecture may be 2D or 3D network and as such, operable to process 2D data and 3D data. Further in paragraph [0135]-LI discloses the carpet view or lines projections 192 are used to create a tissue map 198. The carpet view includes 3D data. Please also read paragraph [0142]) generated by applying an edge extraction filter to classification image data (Fig. 2A-H. Paragraph [0135]-LI discloses a binary morphological reconstruct filter is applied to media and calcium carpet view image 198. Please also read paragraph [0236]) generated based on the three-dimensional classification data (Fig. 2A-H. Paragraph [0134]-LI discloses after media and calcium detection process, each frame in polar space will have corresponding frames/masks for media M and calcium Ca regions/features of interest. These frames/masks may include lumen L and other classes that were used to train neural network for ROI/FOI detection. A set of four output image masks/frames 190 is shown. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. The carpet view or lines projections 192 are used to create a tissue map 198).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range to three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
Wherein BUCKLER’s non-transitory computer-readable medium having further comprising: generating the thick boundary surface data by giving a thickness within the predetermined range to three-dimensional edge data generated by applying an edge extraction filter to classification image data generated based on the three-dimensional classification data.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Regarding claim 20, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the non-transitory computer-readable medium according to claim 19, BUCKLER fails to explicitly teach further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
However, LI explicitly teaches further comprising: generating the thick boundary surface data (Fig. 2A-H. Paragraph [0134]-LI discloses after media and calcium detection process, each frame in polar space will have corresponding frames/masks for media M and calcium Ca regions/features of interest. These frames/masks may include lumen L and other classes that were used to train neural network for ROI/FOI detection. A set of four output image masks/frames 190 is shown. All of the frames of media and calcium mask, which also includes lumen, are projected to get a line of mask. The lines resulting from projection operation are shown as four lines 193. The color coding of pixels in projected lines can be seen with green for media and red for calcium. All the lines are combined into one binary media mask and binary calcium mask. In paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. The carpet view or lines projections 192 are used to create a tissue map 198) by applying a three-dimensional expansion filter to the edge data (Fig. 2A. Paragraph [0135]-LI discloses this combination of line projections 193 is shown as carpet view 195. A binary morphological reconstruct filter is applied to media and calcium carpet view image 198 (wherein a binary morphological reconstruct filter includes opening, closing, dilation and erosion). Please also read paragraph [0220 and 0229]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having a non-transitory computer-readable medium storing a computer program that causes a computer to execute processing, the processing comprising: acquiring classification data in which pixels constituting biomedical image data indicating an internal structure of a living body are classified into a plurality of regions including a living tissue region in which a luminal region exists, the luminal region, and an extraluminal region outside the living tissue region, with the teachings of LI of having further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
Wherein BUCKLER’s non-transitory computer-readable medium having further comprising: generating the thick boundary surface data by applying a three-dimensional expansion filter to the edge data.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and LI concern neural network systems and medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while LI’s systems and methods provide improved classification, information management and diagnostic accuracy of components of medical imaging data. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and LI et al. (US 20200226422 A1), Abstract and Paragraph [0074].
Claims 21, 23 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over BUCKLER et al. (US 20210390689 A1), hereinafter referenced as BUCKLER in view of LI et al. (US 20200226422 A1), hereinafter referenced as LI and in further view of ROLLINS et al. (US 20120075638 A1), hereinafter referenced as ROLLINS and further in view of MULLICK et al. (US 20040101183 A1), hereinafter referenced as MULLICK.
Regarding claim 21, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing method according to claim 1, BUCKLER in view of LI fails to explicitly teach wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
However, MULICK explicitly teaches wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region (Fig. 6. Paragraph [0067]-MULLICK discloses the lumen of a contrast-enhanced vessel 74 may have a substantially uniform intensity relative to bone 72 which may have high intensity on the periphery due to the presence of cortical bone 94 and low intensity on the interior due to the presence of trabecular bone 96. In paragraph [0069]-MULLICK discloses upon calculation of the various statistics, a sequential rule-based classifier is applied at step 120 of FIG. 6 to the statistical results for each region to classify each region by structure type, such as vascular and bone regions. The classification rules may include rules for classifying stented and calcified vessels, rules for identifying vessels, rules for identify bone, and rules for removing small regions labeled as bone which are too small to be bone or which are located where bone is generally not present. In paragraph [0072]-MULLICK discloses the classification of the labeled regions of the output mask 128 may be further assessed by comparison to the neighboring regions in adjacent or proximate slices. Further in paragraph [0078]-MULLICK discloses the bone mask, or other structure mask, may be shown in varying degrees of opacity and translucence)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an information processing method, with the teachings of MULICK of having wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
Wherein BUCKLER’s method having wherein the mask makes the pixel group classified as the living tissue region as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and MULICK concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while MULICK’s systems and methods provide improved techniques for deriving a structure mask with little or no human intervention. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and MULLICK et al. (US 20040101183 A1), Abstract and Paragraph [0008].
Regarding claim 23, BUCKLER in view of LI and in further view of ROLLINS explicitly teach the information processing apparatus according to claim 14, BUCKLER fails to explicitly teach wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region including the luminal region and the extraluminal region.
However, MULICK explicitly teaches wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region including the luminal region and the extraluminal region (Fig. 6. Paragraph [0067]-MULLICK discloses the lumen of a contrast-enhanced vessel 74 may have a substantially uniform intensity relative to bone 72 which may have high intensity on the periphery due to the presence of cortical bone 94 and low intensity on the interior due to the presence of trabecular bone 96. In paragraph [0069]-MULLICK discloses upon calculation of the various statistics, a sequential rule-based classifier is applied at step 120 of FIG. 6 to the statistical results for each region to classify each region by structure type, such as vascular and bone regions. The classification rules may include rules for classifying stented and calcified vessels, rules for identifying vessels, rules for identify bone, and rules for removing small regions labeled as bone which are too small to be bone or which are located where bone is generally not present. In paragraph [0072]-MULLICK discloses the classification of the labeled regions of the output mask 128 may be further assessed by comparison to the neighboring regions in adjacent or proximate slices. Further in paragraph [0078]-MULLICK discloses the bone mask, or other structure mask, may be shown in varying degrees of opacity and translucence).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an information processing method, with the teachings of MULICK of having wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region including the luminal region and the extraluminal region.
Wherein BUCKLER’s method having wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region including the luminal region and the extraluminal region.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and MULICK concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while MULICK’s systems and methods provide improved techniques for deriving a structure mask with little or no human intervention. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and MULLICK et al. (US 20040101183 A1), Abstract and Paragraph [0008].
Regarding claim 25, BUCKLER in view of LI and in further view of ROLLINS teach the non-transitory computer-readable medium according to claim 16, BUCKLER in view of LI fails to explicitly teach wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
However, MULICK explicitly teaches wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region (Fig. 6. Paragraph [0067]-MULLICK discloses the lumen of a contrast-enhanced vessel 74 may have a substantially uniform intensity relative to bone 72 which may have high intensity on the periphery due to the presence of cortical bone 94 and low intensity on the interior due to the presence of trabecular bone 96. In paragraph [0069]-MULLICK discloses upon calculation of the various statistics, a sequential rule-based classifier is applied at step 120 of FIG. 6 to the statistical results for each region to classify each region by structure type, such as vascular and bone regions. The classification rules may include rules for classifying stented and calcified vessels, rules for identifying vessels, rules for identify bone, and rules for removing small regions labeled as bone which are too small to be bone or which are located where bone is generally not present. In paragraph [0072]-MULLICK discloses the classification of the labeled regions of the output mask 128 may be further assessed by comparison to the neighboring regions in adjacent or proximate slices. Further in paragraph [0078]-MULLICK discloses the bone mask, or other structure mask, may be shown in varying degrees of opacity and translucence).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS of having an non-transitory computer-readable medium, with the teachings of MULICK of having wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
Wherein BUCKLER’s non-transitory computer-readable medium having wherein the mask makes the pixel group classified as the living tissue region in the biomedical image data transparent, and makes a pixel group classified as a non-living tissue region opaque, the non-living tissue region including the luminal region and the extraluminal region.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and MULICK concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while MULICK’s systems and methods provide improved techniques for deriving a structure mask with little or no human intervention. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and MULLICK et al. (US 20040101183 A1), Abstract and Paragraph [0008].
Claim 22, 24 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over BUCKLER et al. (US 20210390689 A1), hereinafter referenced as BUCKLER in view of LI et al. (US 20200226422 A1), hereinafter referenced as LI and in further view of ROLLINS et al. (US 20120075638 A1), hereinafter referenced as ROLLINS and further in view of MULLICK et al. (US 20040101183 A1), hereinafter referenced as MULLICK and further in view of KIMMEL et al. (US 20080107315 A1), hereinafter referenced as KIMMEL and further in view of BLABER et al. (WO 2022133200 A1), hereinafter referenced as BLABER.
Regarding claim 22, BUCKLER in view of LI and in further view of ROLLINS and in further view of MULICK explicitly teach the information processing method according to claim 21, BUCKLER in view of LI and in further view of MULICK fails to explicitly teach wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are sent to 0.
However, ROLLINS explicitly teaches wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are sent to 0 (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask). Please also read paragraph [0097]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS of having an information processing method, with the teachings of ROLLINS of having wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are sent to 0.
Wherein BUCKLER’s method having wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are sent to 0.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS fails to explicitly teach generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
However, KIMMEL explicitly teaches generating a mask matrix M (Fig. 1. Paragraph [0039]-KIMMEL discloses the method detects the inner (luminal) and outer (medial-adventitial) boundaries of the vessel, and calcium deposits in the plaque region between the inner and outer boundaries. The term "segmentation" refers to the identification of these features. The method calculates temporal statistics, primarily average intensity and total variation parameters, based on which pixels are classified as either blood or tissue. The inner boundary of the vessel is extracted from the blood/tissue classification. The outer boundary of the vessel is detected by dynamic programming (wherein a blood mask is generated, each row/column of pixels are assigned a true/false value based on a threshold, and the method may further include procedures for eliminating uninformative regions of an IVUS sequence). Please also read paragraph [0071-0072, 0119-0128, and 0139]) in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0 (Fig. 1. Paragraph [0069]-KIMMEL discloses in procedure 174, a tentative assignment of calcium pixels is made for each image based on pixel intensity characteristics. A pixel is assigned "true" (i.e., a calcified plaque pixel) if and only if the intensity of the pixel in the morphologically closed image is high (above a threshold value), and the mean intensity of the pixels above the pixel in the original image is low (below a threshold value). In paragraph [0071]-KIMMEL discloses in procedure 178, the boundary of the calcified plaque regions is determined. The boundary is determined by finding, in each column, the row number of the lowest "true" pixel in the dilated binary image. The lowest "true" pixel represents the pixel which is closest to the catheter. If there is no plaque in a column, a value of zero is assigned. In paragraph [0121]-KIMMEL discloses a "classification table" is developed for determining whether a given combination of average intensity and total variation is more likely to represent a blood pixel or a tissue pixel (wherein the table represents a difference between the lumen (blood) and the media (tissue) histogram). In paragraph [0125]-KIMMEL discloses reference is made to FIG. 11A, which is a binary image demonstrating the application of a threshold on a classification table. In paragraph [0128]-KIMMEL discloses the resulting blood mask is used in the subsequent extraction of the inner and outer boundaries in the IVUS image sequence). Please also read paragraph [0079, 0121-0128 and 0139]) Please also read paragraph [0072, 0079, 0128 and 0139]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS of having an information processing method, with the teachings of KIMMEL of having generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
Wherein BUCKLER’s method having generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and KIMMEL concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while KIMMEL’s systems and methods provide improve the generation of a blood mask for extraction of outer/inner boundaries of a lumen, the statistical blood detector running time and the accuracy of bifurcation sector and uninformative region identification. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and KIMMEL et al. (US 20080107315 A1), Abstract and Paragraph [0128-0129 and 0149].
BUCKLER in view of LI fail to explicitly teach BLUCKNER in view of LI and in further view of ROLLINS and in further view of KIMMEL fail to explicitly teach calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
However, BLABER explicitly teaches calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M (Fig. 2A-H. Paragraph [0068]-BLABER discloses Figures 2A-2F illustrate an example sequence of image frames obtained when the intravascular imaging probe is held stationary at a location within the blood vessel. In paragraph [0079]-BLABER discloses Figure 3 illustrates an example chart identifying the threshold of the segmented images. In paragraph [0083]-BLABER discloses image frame 400A may include the lumen boundary 402, catheter 404, guidewire 406, and bolus 408. In paragraph [0084]-BLABER discloses after the lumen, catheter, and guidewire offsets are determined, the vessel mask 410 may be determined. As shown in image frame 400B, vessel mask 410 may be defined by the boundary of the lumen, guidewire, and catheter), such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R (Fig. 2A-H. Paragraph [0088]-BLABER discloses a contrast mask may be determined for each image frame 200A-200F. The contrast mask may be determined using an elementwise operation between the thresholded image and the vessel mask).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS and in further view of KIMMEL of having an information processing method, with the teachings of BLABER of having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
Wherein BUCKLER’s method having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
The motivation behind the modification would have been to obtain a method that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and BLABER concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while BLABER’s systems and methods provide improve the efficiency of diagnosing the microvascular disease in a patient. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and BLABER et al. (WO 2022133200 A1), Abstract and Paragraph [0029 and 0096].
Regarding claim 24, BUCKLER in view of LI and in further view of MULICK explicitly teach the information processing apparatus according claim 24, BUCKLER in view of LI and in further view of MULICK fail to explicitly teach wherein the generation unit is configured to apply the mask to the thick-line edge data by: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
However, ROLLINS explicitly teaches wherein the generation unit is configured to apply the mask to the thick-line edge data by: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0 (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask). Please also read paragraph [0097]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS of having an information processing apparatus, with the teachings of ROLLINS of having wherein the generation unit is configured to apply the mask to the thick-line edge data by: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
Wherein BUCKLER’s information processing apparatus having wherein the generation unit is configured to apply the mask to the thick-line edge data by: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
The motivation behind the modification would have been to obtain an information processing apparatus that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
BUCKLER in view of LI and in further view of ROLLINS and in further view of MULICK fails to explicitly teach generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
However, KIMMEL explicitly teaches generating a mask matrix M (Fig. 1. Paragraph [0039]-KIMMEL discloses the method detects the inner (luminal) and outer (medial-adventitial) boundaries of the vessel, and calcium deposits in the plaque region between the inner and outer boundaries. The method calculates temporal statistics, primarily average intensity and total variation parameters, based on which pixels are classified as either blood or tissue. The inner boundary of the vessel is extracted from the blood/tissue classification. The outer boundary of the vessel is detected by dynamic programming (wherein a blood mask is generated, each row/column of pixels are assigned a true/false value based on a threshold, and the method may further include procedures for eliminating uninformative regions of an IVUS sequence). Please also read paragraph [0071-0072, 0119-0128, and 0139]) in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0 (Fig. 1. Paragraph [0069]-KIMMEL discloses in procedure 174, a tentative assignment of calcium pixels is made for each image based on pixel intensity characteristics. A pixel is assigned "true" (i.e., a calcified plaque pixel) if and only if the intensity of the pixel in the morphologically closed image is high (above a threshold value), and the mean intensity of the pixels above the pixel in the original image is low (below a threshold value). In paragraph [0071]-KIMMEL discloses in procedure 178, the boundary of the calcified plaque regions is determined. The boundary is determined by finding, in each column, the row number of the lowest "true" pixel in the dilated binary image. The lowest "true" pixel represents the pixel which is closest to the catheter. If there is no plaque in a column, a value of zero is assigned. In paragraph [0121]-KIMMEL discloses a "classification table" is developed for determining whether a given combination of average intensity and total variation is more likely to represent a blood pixel or a tissue pixel (wherein the table represents a difference between the lumen (blood) and the media (tissue) histogram). In paragraph [0125]-KIMMEL discloses reference is made to FIG. 11A, which is a binary image demonstrating the application of a threshold on a classification table. In paragraph [0128]-KIMMEL discloses the resulting blood mask is used in the subsequent extraction of the inner and outer boundaries in the IVUS image sequence). Please also read paragraph [0079, 0121-0128 and 0139]) Please also read paragraph [0072, 0079, 0128 and 0139]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS of having an information processing apparatus, with the teachings of KIMMEL of having wherein generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
Wherein BUCKLER’s information processing apparatus having generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
The motivation behind the modification would have been to obtain an information processing apparatus that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and KIMMEL concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while KIMMEL’s systems and methods provide improve the generation of a blood mask for extraction of outer/inner boundaries of a lumen, the statistical blood detector running time and the accuracy of bifurcation sector and uninformative region identification. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and KIMMEL et al. (US 20080107315 A1), Abstract and Paragraph [0128-0129 and 0149].
BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS and in further view of KIMMEL fail to explicitly teach calculating a region counter matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
However, BLABER explicitly teaches calculating a region counter matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M (Fig. 2A-H. Paragraph [0068]-BLABER discloses Figures 2A-2F illustrate an example sequence of image frames obtained when the intravascular imaging probe is held stationary at a location within the blood vessel. In paragraph [0079]-BLABER discloses Figure 3 illustrates an example chart identifying the threshold of the segmented images. In paragraph [0083]-BLABER discloses image frame 400A may include the lumen boundary 402, catheter 404, guidewire 406, and bolus 408. In paragraph [0084]-BLABER discloses after the lumen, catheter, and guidewire offsets are determined, the vessel mask 410 may be determined. As shown in image frame 400B, vessel mask 410 may be defined by the boundary of the lumen, guidewire, and catheter), such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R (Fig. 2A-H. Paragraph [0088]-BLABER discloses a contrast mask may be determined for each image frame 200A-200F. The contrast mask may be determined using an elementwise operation between the thresholded image and the vessel mask).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS and in further view of KIMMEL of having a non-transitory computer-readable medium, with the teachings of BLABER of having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
Wherein BUCKLER’s non-transitory computer-readable medium having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and BLABER concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while BLABER’s systems and methods provide improve the efficiency of diagnosing the microvascular disease in a patient. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and BLABER et al. (WO 2022133200 A1), Abstract and Paragraph [0029 and 0096].
Regarding claim 26, BUCKLER in view of LI and in further view of MULICK explicitly teach the non-transitory computer-readable medium according to claim 25, BUCKLER in view of LI and in further view of MULICK fails to explicitly teach wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
BUCKLER in view of LI and in further view of MULICK fail to explicitly teach wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
However, ROLLINS explicitly teaches wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0 (Fig. 2. Paragraph [0036]-ROLLINS discloses FIG. 3 illustrates an example of method 30 showing the relationships among different image processing modules for segmenting and quantifying blood vessel structure and physiology. Lumen segmentation step 32 serves a basis for lumen quantification step 33 (wherein segmentation finds boundaries that maximize intensity differences using an energy function e(i, j) representing differences of the left and right side of the gray value at row i and column j). Vessel wall segmentation 35 provides regions of interest for a calcified plaque segmentation step 36, macrophage segmentation step 38 and characterization or classification of other types of plaque step. Calcified plaque quantification 37 may be performed once the calcified plaque segmentation step 36 has been performed. Macrophage quantification 39 may be performed once macrophage quantification 39 has been performed. Further in paragraph [0062]-ROLLINS discloses edge detection of CP may be performed by distinguishing 2 types of edges of a CP in a rectangular intravascular OCT image. With reference to FIG. 2, these edges are the inner border ("IB") 24 and outer border ("OB") 25. These names indicate the edge location with respect to the lumen (wherein edge detection in step 50 includes a combined output of a matched filter 55 (7 by 7 matrix), Prewitt operator (3 by 3 matrix) and a binary mask). Please also read paragraph [0097]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS and in further view of MULICK of having a non-transitory computer-readable medium, with the teachings of ROLLINS of having wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
Wherein BUCKLER’s non-transitory computer-readable medium having wherein the applying of the mask to the thick-line edge data comprises: generating a thick-line edge matrix B in which matrix elements corresponding to pixels constituting a thick line in the thick-line edge data are set to 1 and other matrix elements are set to 0.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and ROLLINS concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while ROLLINS’s systems and methods provide improved understanding of the physiology and structure associated with blood vessels, which can greatly improve diagnosis and treatment of patients. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and ROLLINS et al. (US 20120075638 A1), Abstract and Paragraph [0003].
BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS fail to explicitly teach generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
However, KIMMEL explicitly teaches generating a mask matrix M (Fig. 1. Paragraph [0039]-KIMMEL discloses the method detects the inner (luminal) and outer (medial-adventitial) boundaries of the vessel, and calcium deposits in the plaque region between the inner and outer boundaries. The term "segmentation" refers to the identification of these features. The method calculates temporal statistics, primarily average intensity and total variation parameters, based on which pixels are classified as either blood or tissue. The inner boundary of the vessel is extracted from the blood/tissue classification. The outer boundary of the vessel is detected by dynamic programming (wherein a blood mask is generated, each row/column of pixels are assigned a true/false value based on a threshold, and the method may further include procedures for eliminating uninformative regions of an IVUS sequence). Please also read paragraph [0071-0072, 0119-0128, and 0139]) in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0 (Fig. 1. Paragraph [0069]-KIMMEL discloses in procedure 174, a tentative assignment of calcium pixels is made for each image based on pixel intensity characteristics. A pixel is assigned "true" (i.e., a calcified plaque pixel) if and only if the intensity of the pixel in the morphologically closed image is high (above a threshold value), and the mean intensity of the pixels above the pixel in the original image is low (below a threshold value). In paragraph [0071]-KIMMEL discloses in procedure 178, the boundary of the calcified plaque regions is determined. The boundary is determined by finding, in each column, the row number of the lowest "true" pixel in the dilated binary image. The lowest "true" pixel represents the pixel which is closest to the catheter. If there is no plaque in a column, a value of zero is assigned. In paragraph [0121]-KIMMEL discloses a "classification table" is developed for determining whether a given combination of average intensity and total variation is more likely to represent a blood pixel or a tissue pixel (wherein the table represents a difference between the lumen (blood) and the media (tissue) histogram). In paragraph [0125]-KIMMEL discloses reference is made to FIG. 11A, which is a binary image demonstrating the application of a threshold on a classification table. In paragraph [0128]-KIMMEL discloses the resulting blood mask is used in the subsequent extraction of the inner and outer boundaries in the IVUS image sequence). Please also read paragraph [0079, 0121-0128 and 0139]) Please also read paragraph [0072, 0079, 0128 and 0139]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of ROLLINS and in further view of MULICK of having a non-transitory computer-readable medium, with the teachings of KIMMEL of having generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
Wherein BUCKLER’s non-transitory computer-readable medium having generating a mask matrix M in which matrix elements corresponding to pixels classified into the living tissue region are set to 1 and matrix elements corresponding to pixels classified into the non-living tissue region are set to 0.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and KIMMEL concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while KIMMEL’s systems and methods provide improve the generation of a blood mask for extraction of outer/inner boundaries of a lumen, the statistical blood detector running time and the accuracy of bifurcation sector and uninformative region identification. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and KIMMEL et al. (US 20080107315 A1), Abstract and Paragraph [0128-0129 and 0149].
BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS and in further view of KIMMEL fail to explicitly teach calculating a region counter matrix R as a Hadamard product of the thick- line edge matrix B and the mask matrix M, such that Raj = Bij x Mij,the region contour data being generated based on the region contour matrix R.
However, BLABER explicitly teaches calculating a region counter matrix R as a Hadamard product of the thick- line edge matrix B and the mask matrix M (Fig. 2A-H. Paragraph [0068]-BLABER discloses Figures 2A-2F illustrate an example sequence of image frames obtained when the intravascular imaging probe is held stationary at a location within the blood vessel. In paragraph [0079]-BLABER discloses Figure 3 illustrates an example chart identifying the threshold of the segmented images. In paragraph [0083]-BLABER discloses image frame 400A may include the lumen boundary 402, catheter 404, guidewire 406, and bolus 408. In paragraph [0084]-BLABER discloses after the lumen, catheter, and guidewire offsets are determined, the vessel mask 410 may be determined. As shown in image frame 400B, vessel mask 410 may be defined by the boundary of the lumen, guidewire, and catheter), such that Raj = Bij x Mij,the region contour data being generated based on the region contour matrix R (Fig. 2A-H. Paragraph [0088]-BLABER discloses a contrast mask may be determined for each image frame 200A-200F. The contrast mask may be determined using an elementwise operation between the thresholded image and the vessel mask).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BUCKLER in view of LI and in further view of MULICK and in further view of ROLLINS and in further view of KIMMEL of having a non-transitory computer-readable medium, with the teachings of BLABER of having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
Wherein BUCKLER’s non-transitory computer-readable medium having calculating a region contour matrix R as a Hadamard product of the thick-line edge matrix B and the mask matrix M, such that Raj = Bij x Mij, the region contour data being generated based on the region contour matrix R.
The motivation behind the modification would have been to obtain a non-transitory computer-readable medium that improves medical imaging classification, diagnostics and outcome prediction, since both BUCKLER and BLABER concern medical imaging involving a luminal area. Wherein BUCKLER’s systems and methods improve upon both phenotype classification and outcome prediction, while BLABER’s systems and methods provide improve the efficiency of diagnosing the microvascular disease in a patient. Please see BUCKLER et al. (US 20210390689 A1), Abstract and Paragraph [0063 and 0068] and BLABER et al. (WO 2022133200 A1), Abstract and Paragraph [0029 and 0096].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant`s disclosure.
KIMMEL et al. (US 20080107315 A1)- Method for performing segmentation of an interior vessel within the body of a patient, the method including obtaining a sequence of intravascular ultrasound images of an interior vessel and dividing the sequence into batches, detecting uninformative regions in each of the batches, the uninformative regions arising from an acoustic shadow cast by guide wire and by calcified plaque within the interior vessel, extracting a preliminary outer boundary of the interior vessel, tracking images in each of the batches to counter various distortions, performing statistical analysis and spatial integration on each of the batches to obtain a classification of blood and tissue regions, extracting a secondary outer boundary of the interior vessel utilizing the classification of blood and tissue regions and refining result, and extracting the inner boundary of the interior vessel based on the classification of blood and tissue regions.........................Please see Fig. 1-2 and 8, and read para. [0069-0072, 0079 and 00119-0128]. Abstract.
ZHANG (US 20210174125 A1)- Method for performing segmentation of an interior vessel within the body of a patient, the method including obtaining a sequence of intravascular ultrasound images of an interior vessel and dividing the sequence into batches, detecting uninformative regions in each of the batches, the uninformative regions arising from an acoustic shadow cast by guide wire and by calcified plaque within the interior vessel, extracting a preliminary outer boundary of the interior vessel, tracking images in each of the batches to counter various distortions, performing statistical analysis and spatial integration on each of the batches to obtain a classification of blood and tissue regions, extracting a secondary outer boundary of the interior vessel utilizing the classification of blood and tissue regions and refining result, and extracting the inner boundary of the interior vessel based on the classification of blood and tissue regions.........................Please see Fig. 3-4, and read para. [0070, 0090, 0092, and 0122-0134]. Abstract.
HE et al. (US 20180271496 A1)- Method for performing segmentation of an interior vessel within the body of a patient, the method including obtaining a sequence of intravascular ultrasound images of an interior vessel and dividing the sequence into batches, detecting uninformative regions in each of the batches, the uninformative regions arising from an acoustic shadow cast by guide wire and by calcified plaque within the interior vessel, extracting a preliminary outer boundary of the interior vessel, tracking images in each of the batches to counter various distortions, performing statistical analysis and spatial integration on each of the batches to obtain a classification of blood and tissue regions, extracting a secondary outer boundary of the interior vessel utilizing the classification of blood and tissue regions and refining result, and extracting the inner boundary of the interior vessel based on the classification of blood and tissue regions.........................Please see para. [0049-0050]. Abstract.
ZUR et al. (US 20180253839 A1)- An image processing system connected to an endoscope and processing in real-time endoscopic images to identify suspicious tissues such as polyps or cancer. The system applies preprocessing tools to clean the received images and then applies in parallel a plurality of detectors both conventional detectors and models of supervised machine learning-based detectors. A post processing is also applied in order select the regions which are most probable to be suspicious among the detected regions. Frames identified as showing suspicious tissues can be marked on an output video display. Optionally, the size, type and boundaries of the suspected tissue can also be identified and marked..........................Please see para. [0094-0099, 0116-0117, 0150-0155]. Abstract.
LEE et al. (US 20220284584 A1)- Methods for training an algorithm to identify structural anatomical features, for example of a blood vessel, in a non-contrast computed tomography (NCT) image are described herein. The algorithm may comprise an image segmentation algorithm, a random forest classifier, or a generative adversarial network in examples described herein. In one embodiment, a method comprises receiving a labelled training set for a machine learning image segmentation algorithm. The labelled training set comprising a plurality of NCT images, each NCT image of the plurality of NCT images showing a targeted region of a subject, the targeted region including at least one blood vessel. The labelled training set further comprises a corresponding plurality of segmentation masks, each segmentation mask labelling at least one structural feature of a blood vessel in a corresponding NCT image of the plurality of NCT images. The method further comprises training a machine learning image segmentation algorithm, using the plurality of NCT images and the corresponding plurality of segmentation masks, to learn features of the NCT images that correspond to structural features of the blood vessels labelled in the segmentation masks, and output a trained image segmentation model. The method further comprises outputting the trained image segmentation model usable for identifying structural features of a blood vessel in an NCT image. Further methods are described herein for identifying anatomical features from an NCT image, and for establishing training sets. Computing apparatuses and computer readable media are also described herein.........................Please see Fig. 1-4. Abstract.
LU et al. (US 20220175269 A1)- Systems and methods for locating a medical device in a body lumen are provided. A first flexible elongate instrument comprises a plurality of imaging markers, and a location information sensor is disposed at the first flexible elongate instrument or at a second flexible elongate instrument configured for relative movement with respect to the first flexible elongate instrument. A processor is configured to establish a reference coordinate system based on the plurality of imaging markers, which are visible in a medical image comprising the first flexible elongate instrument disposed in a body lumen, receive diagnostic scan or therapeutic delivery information at a plurality of locations of the body lumen from the first or second flexible elongate instrument, and correlate the information with the imaging markers. A display configured to display a composite image comprising the correlated diagnostic scan or therapeutic delivery information and the imaging markers......................Please see Fig. 20 and para. [0228-0240]. Abstract.
HUANG et al. (US 20180344147 A1)- Methods of applying OCT angiography are disclosed. In particular, methods of detecting, visualizing and measuring the extent of retinal neovascularization are disclosed. Further disclosed are methods measuring retinal nonperfusion area and choriocapillaris defect area........................Please see Para. [0136]. Abstract.
Gopinath et al. (US 20200294659 A1)- In part, the disclosure relates to method of displaying a representation of an artery. The method may include storing an intravascular image dataset in a memory device of a diagnostic imaging system, the intravascular image dataset generated in response to intravascular imaging of a segment of an artery; automatically detecting lumen boundary of the segment on a per frame basis; automatically detecting EEL and displaying a stent sizing workflow. In part, the disclosure also relates to automatically detecting one or more regions of calcium relative to lumen boundary of the segment; calculating an angular or circumferential measurement of detected calcium for one or more frames; calculating a calcium thickness of detected calcium for one or more frames; and displaying the calcium thickness and the angular or circumferential measurement of detected calcium for a first frame of the one or more frames....................Please see Fig. 1-8. Abstract.
CHONO (US 20120083698 A1)- Disclosed is an ultrasonic diagnostic apparatus provided with an imaging unit configured to obtain an ultrasonic image of a carotid artery portion of an object to be examined, a parting line calculation unit configured to set at least one threshold value and calculate a parting line for segmenting the ultrasonic image into multiple regions, an intima-media thickness calculation unit configured to specify a direction for searching the multiple regions, search an intima-media region in the specified direction based on the brightness, draw multiple curves based on the position acquired by the search and positional information on the carotid artery portion in the ultrasonic image, and calculate the intima-media thickness from the distance between the multiple curves.....................Please see Fig. 1-10. Abstract.
Madabhushi (US 20210158524 A1)- Embodiments discussed herein facilitate segmentation of histological primitives from stained histology of renal biopsies via deep learning and/or training deep learning model(s) to perform such segmentation. One example embodiment is configured to access a first histological image of a renal biopsy comprising a first type of histological primitives, wherein the first histological image is stained with a first type of stain; provide the first histological image to a first deep learning model trained based on the first type of histological primitive and the first type of stain; and receive a first output image from the first deep learning model, wherein the first type of histological primitives is segmented in the first output image......................Please see Fig. 16 and read paragraph [0077-0082, 0092-0096, 0106 and 0109]. Abstract.
TANG et al. (US 20060149522 A1)- A method and a computer system, for providing an assessment for disease status of a disease, such as a cardiovascular disease, employ construction of image-based 3D computational model of an organ representative of the disease status; computationally obtaining a certain mechanical distribution using the 3D-organ model; and computational, quantitative analysis of the mechanical distribution to provide an assessment for disease status of a disease. The image-based 3D computational model includes a fluid-structure interaction and multiple components within the organ.........................Please see Fig. 1-7. Abstract.
SURI (US 20110299753 A1)- A computer-implemented system and method for fast, reliable, and automated embodiments for using a multi-resolution edge flow approach to vascular ultrasound for intima-media thickness (IMT) measurement. Various embodiments include receiving biomedical imaging data and patient demographic data corresponding to a current scan of a patient; checking the biomedical imaging data in real-time to determine if an artery of the patient has a calcium deposit in a proximal wall of the artery; acquiring arterial data of the patient as a combination of longitudinal B-mode and transverse B-mode data; using a data processor to automatically recognize the artery; using the data processor to calibrate a region of interest around the automatically recognized artery; automatically computing the weak or missing edges of intima-media and media-adventitia walls using edge flow, labeling and connectivity; and determining the intima-media thickness (IMT) of an arterial wall of the automatically recognized artery.......................Please see Para. [0150-0174]. Abstract.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to Aaron Bonansinga whose telephone number is (703) 756-5380 The examiner can normally be reached on Monday-Friday, 9:00 a.m. - 6:00 p.m. ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Chineyere Wills-Burns can be reached by phone at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON TIMOTHY BONANSINGA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights