DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/12/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 3, 5-6, 8-10, 14-18, & 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 3, 5-6, 8-10, 14-18, 20, & 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 20, and 22 recite the following “each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted”; however, no accompanying passage in the specification adequately provides support for the limitation.
Applicants Spec [0094] indicates that the background region is only allotted once all findings in the probability map are below a threshold (“in a case in which the presence probabilities of all of the findings are less than the probability threshold value, the probability map generation unit 10 can determine that the pixels belong to the background region R5 which does not correspond to any findings.”). At no point in the specification is there an indication that for each probability map a background voxel is determined as well as a findings voxel; the specification only provides evidence for a binary decision either the map is a background or a finding.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5-6, 8-10, 14-18, 20, & 22 are rejected under 35 U.S.C. 103 as being unpatentable over Ribes et al (US20160113632A1; hereinafter referred to as Ribes) in view of Mccormac et al (US20190147220A1; hereinafter referred to as Mccormac) and further in view Aoi et al (N. Aoi et al., “Ultrasound assessment of deep tissue injury in pressure ulcers: Possible prediction of pressure ulcer progression,” Plastic and Reconstructive Surgery, vol. 124, no. 2, pp. 540–550, Aug. 2009.;hereinafter referred to as Aoi).
Regarding Claim 1, Kapoor discloses an ultrasound diagnostic apparatus (“FIG. 14 shows a system for three-dimensional ultrasound imaging. The system includes a medical imaging system 11, a transducer 13, a sensor 15, a processor 12, a memory 14, and a display 16. The processor 12 and the memory 14 are shown separate from the medical imaging system 11, such associated with being a computer or workstation apart from the medical imaging system 11.” [0098]) comprising:
an ultrasound probe; a position sensor that is attached to the ultrasound probe and configured to acquire positional information of the ultrasound probe (“The transducer 13 is an array, such as a one dimensional, two dimensional, or multi-dimensional array of elements. The elements are piezoelectric or capacitive membrane ultrasound transducer elements. Any now known or later developed array may be used.” [0100], “The sensor 15 connects with or is separate from the transducer 13. For example, the sensor 15 is a magnetic position sensor with antennas in the transducer 13 as well as antennas spaced from the transducer 13. In another example, the sensor 15 is an optical sensor, such as a camera spaced from the transducer 13. A processor processes images from the optical sensor to determine the position and orientation of the transducer 13.” [0102]);
a processor configured to: scan a region of interest of a subject with an ultrasound beam using the ultrasound probe to acquire B-mode images of a plurality of frames (“The clinician scans the target anatomy in a painting-like motion not being restricted to unidirectional sweeps. To achieve a high-quality 3D reconstruction, the acquired 2D ultrasound images are passed through a processing pipeline recovering orientation and position errors present due to the limited tracking accuracy, and grouping the frames into sequences with uniform orientation. The 2D frames are transformed into 3D space using the orientation and position information, and their image information is compounded into a 3D volume” [0021], “In act 20, data representing a patient is obtained. The data is acquired by scanning the patient. An ultrasound system scans the patient by generating acoustic energy with a transducer and receiving echoes. Any scanning sequence or approach may be used, such as a linear, Vector® or sector scan format of scan lines. The echoes are received by the transducer and converted into electrical signals. The electrical signals are processed to detect intensity (e.g., B-mode), tissue strain, power Doppler, contrast, tissue harmonics, elastic modulus, relaxation time, other tissue property, or other scalar information.” [0030]).
Kapoor does not specifically disclose the region of interest including a wound portion; calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames, the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels.
However, in a similar field of endeavor, McCormac teaches processing frames of video data and labelling surface elements within a representation of a three-dimensional (3D) space [0002].
McCormac also teaches calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames (“Video data 170 may further comprise any measured sensory input that is arranged in a two-dimensional form representative of a captured or recorded view of a 3D space with a capture device. For example, this may comprise just one of depth data or photometric data, electromagnetic imaging, ultrasonic imaging and radar output, amongst others.” [0061], “At block 605, object-label probability values for spatial elements of frames of video data are determined using a two-dimensional image classifier. For example, the two-dimensional image classifier may be a CNN as described above that is configured to receive a frame of video data as a 2D image and to output probability values for at least one area of the 2D image. In this case, the probability values relate to a plurality of available object or class labels, e.g. string labels representing words and/or identifiers for object data definitions. In one case, probability values may be output for each pixel of an input 2D image and this output may be structured as one or more images, wherein in certain cases each image comprises a probability distribution for a particular object label. These output images can be referred to as “pixel maps”. This may be the case where the two-dimensional image classifier comprises a deconvolutional neural network communicatively coupled to an output of a CNN. The two-dimensional image classifier may be configured to compute object-label probability values based on at least colour data and depth data for a frame.” [0100]);
generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings (“In parallel with the operation of components 515 to 560, the image classifier 565 is configured to receive a frame of video data, Ft, and compute a set of object-label probability distributions 570. In this case, as described above, the object-label probability distributions 570 are provided as a set of images, each image corresponding to a different object label, wherein spatial element values, in this case pixel values, in the images represent a probability that an object having the label is visible in the spatial element. In FIG. 5, the set of object-label probability distributions 570 are received by a surfel-to-pixel mapper 575. In this example, the surfel-to-pixel mapper 575 is configured to determine a correspondence between a spatial element (e.g. a pixel) in the images 570 and a surfel in the surfel representation 530. This correspondence is based on the tracked pose, Pt, which may be stored with the surfel representation 530 or supplied by the frame-to-model tracking component 515. Using the tracked pose, Pt, each surfel at a given 3D location wx(s) in the surfel representation 530 may be associated with pixel coordinates u associated with a frame Ft via the camera projection u(s, t)=π(cwPtwx(s)), where cwPt is the pose of the capture device (“C”) with respect to the “world” (“W”) of the 3D representation which is calculated employing the homogeneous transformation matrix cwPt=wcPt −1 (where wcPt is the pose Pt of the “world” with respect to the capture device) and using homogeneous 3D coordinates.” [0091];
and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels (“In FIG. 5, once a set of object-label probability values 580 have been assigned to surfels deemed visible in a frame of video data they are sent to the Bayesian update engine 585. For example, the surfel-to-pixel mapper 575 may be configured to output a set of surfel-probability pairings, identifying a given surfel (e.g. “SF1234”) and the corresponding probability values from the image classifier 565. The Bayesian update engine 585 is configured to retrieve existing object-label probability values, SPt-1, for surfels identified in the data 580 and to use the new probability values in data 580 to update these existing values. For example, Bayesian update engine 585 may be configured to update all the surfels in a visible set from the surfel representation 530 with a corresponding probability distribution by means of a recursive Bayesian update.” [0092]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other.
However, in a similar field of endeavor, Aoi teaches using ultrasound to evaluate deep tissue damage under pressure ulcers, and analyzed its potential usefulness in diagnosing deep tissue in jury at the early stages and predicting the prog nosis of pressure ulcers [Pg. 541 Col. 1].
Aoi also teaches the region of interest including a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other (“An unclear layered structure was observed in all 12 patients followed and seems to be the ultrasonic sign most commonly seen in deep tissue injury. An unclear layered structure likely reflects subcutaneous tissue damage such as edema, interstitial fluid, and inflammation that result in a coarse echo texture and the unclear margins of each layer in echoic images.” [Pg. 546 Col. 2], “The image of a heterogeneous hypoechoic area would represent a necrotic area containing necrotic tissue, debris, and fluid; the necrotic area shows a heterogeneous area on ultrasound, with relatively low internal echo.” [Pg. 548 Col. 1]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 3, Kapoor discloses the processor is further comprising: a monitor (“The display 16 is a CRT, LCD, plasma, projector, printer, or other output device for showing an image. The display 16 displays an image of the volume of the patient. A measure based on the scanned volume, such as a distance, may be displayed in a chart, graph, and/or on an image. The image may include other information, such as an overlay or color coding from the indications.” [0111]).
Kapoor does not specifically disclose that the processor is further configured to display the integrated probability map on the monitor.
However, in a similar field of endeavor, McCormac teaches the processor is further configured to display the integrated probability map on the monitor (“object-label probability values for the surfels in the 3D surfel representation are updated based on the object-label probability values for corresponding spatial elements. This results in a semantically-labelled 3D surfel representation of objects present in the video data. For example, an estimate representing objects present in the video data may be generated by taking the highest probability value in the set of object label probability values for each surfel (e.g. using a maximum likelihood approach).” [0102])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the processor is further configured to display the integrated probability map on the monitor as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Regarding Claim 5, Kapoor discloses the processor is further configured to: calculate finding information including at least one of a size, a depth, or a volume of the multiple types of findings based on the three-dimensional probability maps; and display the finding information on the monitor (“a representation of the volume is displayed as one or more two-dimensional (e.g., 3D renderings, volume cross sections) images on the display. The displayed images may be arbitrary reformats of the volume. The reformatting restores the curved swept surface of the skin or physical structure to a plane or the curved path to a straight line. A user interface allows the user to interact with the display to perform quantitative measurements of volume, area, length, average intensity, intensity profile or other measurement. The user may define any measurement, such as selecting an arbitrary group of voxels. The processor or user may classify different voxels or pixels as belonging to different categories (e.g., types of tissue or lesion). The measurements may be made specific to the category.” [0096]).
Regarding Claim 6, Kapoor discloses the processor is further configured to generate the integrated probability map in which a shade of a color is changed according to a value of the presence probabilities (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
Regarding Claim 8, Kapoor discloses the processor is further configured to: perform a plurality of scanning operations on the region of interest to calculate the presence probabilities in each of the plurality of scanning operations (“Due to overlap, orientation change, or rescanning a same location at different times (e.g., sweeping the transducer back and forth), data of one frame may represent a same location as data of another frame. A same volume may be represented by different sets of ultrasound data. A given set of data representing a volume may extend beyond the same volume, such as representing a location or locations different than for another volume.” [0033]).
Kapoor does not specifically disclose generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations.
However, in a similar field of endeavor, McCormac teaches generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations (“the correspondence interface 230 is configured to provide an updated correspondence following a spatial deformation of the surfel representation 270. This may be a non-rigid spatial deformation using a deformation graph, wherein the surfels in the surfel representation form the nodes of said graph. The spatial deformation enacts a loop closure within the video data. For example, this may relate to a capture device “re-observing” a particular part of a scene or space, e.g. viewing objects that are modelled in the surfel representation a second time or from a different angle. In the present example, the semantic augmenter 240 uses the updated correspondence to update object-label probability values for a first surfel using object-label probability values for spatial elements that previously corresponded to a second surfel. In other words, following a loop closure event, the spatial deformation modifies the 3D position of the surfels meaning that in an update operation for a subsequent frame without movement of the capture device, the correspondence data 260 is different and as such different sets of surfel probability values are updated despite object-label probability distributions 250 remaining the same due to the lack of movement.” [0067])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the processor is further configured to display the integrated probability map on the monitor as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach that the region of interest is a wound portion.
However, in a similar field of endeavor, Aoi teaches the region of interest is a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest is a wound portion as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 9, Kapoor discloses the processor is further configured to: perform a plurality of scanning operations on the region of interest to calculate the presence probabilities in each of the plurality of scanning operations; and generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations (“Due to overlap, orientation change, or rescanning a same location at different times (e.g., sweeping the transducer back and forth), data of one frame may represent a same location as data of another frame. A same volume may be represented by different sets of ultrasound data. A given set of data representing a volume may extend beyond the same volume, such as representing a location or locations different than for another volume.” [0033]).
Kapoor does not specifically disclose generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations.
However, in a similar field of endeavor, McCormac teaches generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations (“the correspondence interface 230 is configured to provide an updated correspondence following a spatial deformation of the surfel representation 270. This may be a non-rigid spatial deformation using a deformation graph, wherein the surfels in the surfel representation form the nodes of said graph. The spatial deformation enacts a loop closure within the video data. For example, this may relate to a capture device “re-observing” a particular part of a scene or space, e.g. viewing objects that are modelled in the surfel representation a second time or from a different angle. In the present example, the semantic augmenter 240 uses the updated correspondence to update object-label probability values for a first surfel using object-label probability values for spatial elements that previously corresponded to a second surfel. In other words, following a loop closure event, the spatial deformation modifies the 3D position of the surfels meaning that in an update operation for a subsequent frame without movement of the capture device, the correspondence data 260 is different and as such different sets of surfel probability values are updated despite object-label probability distributions 250 remaining the same due to the lack of movement.” [0067])
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Kapoor as outlined above with the processor is further configured to display the integrated probability map on the monitor as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach that the region of interest is a wound portion.
However, in a similar field of endeavor, Aoi teaches the region of interest is a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest is a wound portion as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 10, Kapoor discloses the processor is further configured to: perform a plurality of scanning operations on the region of interest to calculate the presence probabilities in each of the plurality of scanning operations; and generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations (“Due to overlap, orientation change, or rescanning a same location at different times (e.g., sweeping the transducer back and forth), data of one frame may represent a same location as data of another frame. A same volume may be represented by different sets of ultrasound data. A given set of data representing a volume may extend beyond the same volume, such as representing a location or locations different than for another volume.” [0033]).
Kapoor does not specifically disclose generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations.
However, in a similar field of endeavor, McCormac teaches generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations (“the correspondence interface 230 is configured to provide an updated correspondence following a spatial deformation of the surfel representation 270. This may be a non-rigid spatial deformation using a deformation graph, wherein the surfels in the surfel representation form the nodes of said graph. The spatial deformation enacts a loop closure within the video data. For example, this may relate to a capture device “re-observing” a particular part of a scene or space, e.g. viewing objects that are modelled in the surfel representation a second time or from a different angle. In the present example, the semantic augmenter 240 uses the updated correspondence to update object-label probability values for a first surfel using object-label probability values for spatial elements that previously corresponded to a second surfel. In other words, following a loop closure event, the spatial deformation modifies the 3D position of the surfels meaning that in an update operation for a subsequent frame without movement of the capture device, the correspondence data 260 is different and as such different sets of surfel probability values are updated despite object-label probability distributions 250 remaining the same due to the lack of movement.” [0067]).
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Kapoor as outlined above with generate each of the three-dimensional probability maps based on the presence probabilities which are calculated in the plurality of scanning operations as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach that the region of interest is a wound portion.
However, in a similar field of endeavor, Aoi teaches the region of interest is a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest is a wound portion as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 14, Kapoor discloses the processor is further configured to: analyze the integrated probability map or the B-mode images to detect a failure region in which at least one of the three-dimensional probability maps is not normally generated or at least one of the B-mode images is not normally acquired; and display the failure region on the monitor (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
Regarding Claim 15, Kapoor discloses the processor is further configured to:analyze the integrated probability map or the B-mode images to detect a failure region in which at least one of the three-dimensional probability maps is not normally generated or at least one of the B-mode images is not normally acquired; anddisplay the failure region on the monitor (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
Regarding Claim 16, Kapoor discloses the processor is further configured to: detect that a region has already been scanned based on the positional information of the ultrasound probe acquired by the position sensor in a case in which scanning is performed on the region of interest; and notify a user that the region has already been scanned (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
Kapoor in view of McCormac does not specifically teach that the region of interest is a wound portion.
However, in a similar field of endeavor, Aoi teaches the region of interest is a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest is a wound portion as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 17, Kapoor discloses the processor is further configured to display the map that has already been generated on the monitor based on the positional information of the ultrasound probe acquired by the position sensor in a case in which scanning is performed on the region of interest (“In act 24, the frames are spatially registered. The spatial registration is relative to the volume and/or other frames. The frames are spatially registered with each other so that the proper or actual position of the planes represented by the frames is located in the volume.” [0048], “In act 26, the transform based on tracking signals is altered or a further transformation matrix is included to increase the accuracy of the tracking signals. The locations from planes represented by the frames are transformed in 3D space using the signals from the sensor on the transducer. These signals are filtered to increase accuracy or remove noise. For example, the signals are low pass filtered. The sensors of position as a function of time are temporally or spatially filtered. Any low pass filtering may be used.” [0051]).
Kapoor does not specifically disclose display the integrated probability map.
However, in a similar field of endeavor, McCormac teaches display the integrated probability map (“the correspondence interface 230 is configured to provide an updated correspondence following a spatial deformation of the surfel representation 270. This may be a non-rigid spatial deformation using a deformation graph, wherein the surfels in the surfel representation form the nodes of said graph. The spatial deformation enacts a loop closure within the video data. For example, this may relate to a capture device “re-observing” a particular part of a scene or space, e.g. viewing objects that are modelled in the surfel representation a second time or from a different angle. In the present example, the semantic augmenter 240 uses the updated correspondence to update object-label probability values for a first surfel using object-label probability values for spatial elements that previously corresponded to a second surfel. In other words, following a loop closure event, the spatial deformation modifies the 3D position of the surfels meaning that in an update operation for a subsequent frame without movement of the capture device, the correspondence data 260 is different and as such different sets of surfel probability values are updated despite object-label probability distributions 250 remaining the same due to the lack of movement.” [0067]).
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Kapoor as outlined above with display the integrated probability map as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach that the region of interest is a wound portion.
However, in a similar field of endeavor, Aoi teaches the region of interest is a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest is a wound portion as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 18, Kapoor discloses the processor is further configured to generate the map in which a region, which has been repeatedly scanned a larger number of times, has a darker color or a higher density (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
Kapoor does not specifically teach generating the integrated probability map.
However, in a similar field of endeavor, McCormac teaches generating the integrated probability map (“the correspondence interface 230 is configured to provide an updated correspondence following a spatial deformation of the surfel representation 270. This may be a non-rigid spatial deformation using a deformation graph, wherein the surfels in the surfel representation form the nodes of said graph. The spatial deformation enacts a loop closure within the video data. For example, this may relate to a capture device “re-observing” a particular part of a scene or space, e.g. viewing objects that are modelled in the surfel representation a second time or from a different angle. In the present example, the semantic augmenter 240 uses the updated correspondence to update object-label probability values for a first surfel using object-label probability values for spatial elements that previously corresponded to a second surfel. In other words, following a loop closure event, the spatial deformation modifies the 3D position of the surfels meaning that in an update operation for a subsequent frame without movement of the capture device, the correspondence data 260 is different and as such different sets of surfel probability values are updated despite object-label probability distributions 250 remaining the same due to the lack of movement.” [0067]).
It would have been obvious to an ordinary skilled person in the art before the effective filing date of the claimed invention to modify the system of Kapoor as outlined above with generating the integrated probability map as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Regarding Claim 20, Kapoor discloses a processor for an ultrasound diagnostic apparatus (“FIG. 14 shows a system for three-dimensional ultrasound imaging. The system includes a medical imaging system 11, a transducer 13, a sensor 15, a processor 12, a memory 14, and a display 16. The processor 12 and the memory 14 are shown separate from the medical imaging system 11, such associated with being a computer or workstation apart from the medical imaging system 11.” [0098]) the processor being configured to:
acquire positional information of an ultrasound probe (“The transducer 13 is an array, such as a one dimensional, two dimensional, or multi-dimensional array of elements. The elements are piezoelectric or capacitive membrane ultrasound transducer elements. Any now known or later developed array may be used.” [0100], “The sensor 15 connects with or is separate from the transducer 13. For example, the sensor 15 is a magnetic position sensor with antennas in the transducer 13 as well as antennas spaced from the transducer 13. In another example, the sensor 15 is an optical sensor, such as a camera spaced from the transducer 13. A processor processes images from the optical sensor to determine the position and orientation of the transducer 13.” [0102]);
scan a region of interest of a subject with an ultrasound beam using the ultrasound probe to acquire B-mode images of a plurality of frames (“The clinician scans the target anatomy in a painting-like motion not being restricted to unidirectional sweeps. To achieve a high-quality 3D reconstruction, the acquired 2D ultrasound images are passed through a processing pipeline recovering orientation and position errors present due to the limited tracking accuracy, and grouping the frames into sequences with uniform orientation. The 2D frames are transformed into 3D space using the orientation and position information, and their image information is compounded into a 3D volume” [0021], “In act 20, data representing a patient is obtained. The data is acquired by scanning the patient. An ultrasound system scans the patient by generating acoustic energy with a transducer and receiving echoes. Any scanning sequence or approach may be used, such as a linear, Vector® or sector scan format of scan lines. The echoes are received by the transducer and converted into electrical signals. The electrical signals are processed to detect intensity (e.g., B-mode), tissue strain, power Doppler, contrast, tissue harmonics, elastic modulus, relaxation time, other tissue property, or other scalar information.” [0030]).
Kapoor does not specifically disclose the region of interest including a wound portion; calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames, the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels.
However, in a similar field of endeavor, McCormac teaches processing frames of video data and labelling surface elements within a representation of a three-dimensional (3D) space [0002].
McCormac also teaches calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames (“Video data 170 may further comprise any measured sensory input that is arranged in a two-dimensional form representative of a captured or recorded view of a 3D space with a capture device. For example, this may comprise just one of depth data or photometric data, electromagnetic imaging, ultrasonic imaging and radar output, amongst others.” [0061], “At block 605, object-label probability values for spatial elements of frames of video data are determined using a two-dimensional image classifier. For example, the two-dimensional image classifier may be a CNN as described above that is configured to receive a frame of video data as a 2D image and to output probability values for at least one area of the 2D image. In this case, the probability values relate to a plurality of available object or class labels, e.g. string labels representing words and/or identifiers for object data definitions. In one case, probability values may be output for each pixel of an input 2D image and this output may be structured as one or more images, wherein in certain cases each image comprises a probability distribution for a particular object label. These output images can be referred to as “pixel maps”. This may be the case where the two-dimensional image classifier comprises a deconvolutional neural network communicatively coupled to an output of a CNN. The two-dimensional image classifier may be configured to compute object-label probability values based on at least colour data and depth data for a frame.” [0100]);
generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings (“In parallel with the operation of components 515 to 560, the image classifier 565 is configured to receive a frame of video data, Ft, and compute a set of object-label probability distributions 570. In this case, as described above, the object-label probability distributions 570 are provided as a set of images, each image corresponding to a different object label, wherein spatial element values, in this case pixel values, in the images represent a probability that an object having the label is visible in the spatial element. In FIG. 5, the set of object-label probability distributions 570 are received by a surfel-to-pixel mapper 575. In this example, the surfel-to-pixel mapper 575 is configured to determine a correspondence between a spatial element (e.g. a pixel) in the images 570 and a surfel in the surfel representation 530. This correspondence is based on the tracked pose, Pt, which may be stored with the surfel representation 530 or supplied by the frame-to-model tracking component 515. Using the tracked pose, Pt, each surfel at a given 3D location wx(s) in the surfel representation 530 may be associated with pixel coordinates u associated with a frame Ft via the camera projection u(s, t)=π(cwPtwx(s)), where cwPt is the pose of the capture device (“C”) with respect to the “world” (“W”) of the 3D representation which is calculated employing the homogeneous transformation matrix cwPt=wcPt −1 (where wcPt is the pose Pt of the “world” with respect to the capture device) and using homogeneous 3D coordinates.” [0091];
and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels (“In FIG. 5, once a set of object-label probability values 580 have been assigned to surfels deemed visible in a frame of video data they are sent to the Bayesian update engine 585. For example, the surfel-to-pixel mapper 575 may be configured to output a set of surfel-probability pairings, identifying a given surfel (e.g. “SF1234”) and the corresponding probability values from the image classifier 565. The Bayesian update engine 585 is configured to retrieve existing object-label probability values, SPt-1, for surfels identified in the data 580 and to use the new probability values in data 580 to update these existing values. For example, Bayesian update engine 585 may be configured to update all the surfels in a visible set from the surfel representation 530 with a corresponding probability distribution by means of a recursive Bayesian update.” [0092]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other.
However, in a similar field of endeavor, Aoi teaches using ultrasound to evaluate deep tissue damage under pressure ulcers, and analyzed its potential usefulness in diagnosing deep tissue in jury at the early stages and predicting the prog nosis of pressure ulcers [Pg. 541 Col. 1].
Aoi also teaches the region of interest including a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other (“An unclear layered structure was observed in all 12 patients followed and seems to be the ultrasonic sign most commonly seen in deep tissue injury. An unclear layered structure likely reflects subcutaneous tissue damage such as edema, interstitial fluid, and inflammation that result in a coarse echo texture and the unclear margins of each layer in echoic images.” [Pg. 546 Col. 2], “The image of a heterogeneous hypoechoic area would represent a necrotic area containing necrotic tissue, debris, and fluid; the necrotic area shows a heterogeneous area on ultrasound, with relatively low internal echo.” [Pg. 548 Col. 1]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Regarding Claim 22, Kapoor discloses an ultrasound diagnostic apparatus (“FIG. 14 shows a system for three-dimensional ultrasound imaging. The system includes a medical imaging system 11, a transducer 13, a sensor 15, a processor 12, a memory 14, and a display 16. The processor 12 and the memory 14 are shown separate from the medical imaging system 11, such associated with being a computer or workstation apart from the medical imaging system 11.” [0098]) comprising:
an ultrasound probe; a position sensor that is attached to the ultrasound probe and configured to acquire positional information of the ultrasound probe (“The transducer 13 is an array, such as a one dimensional, two dimensional, or multi-dimensional array of elements. The elements are piezoelectric or capacitive membrane ultrasound transducer elements. Any now known or later developed array may be used.” [0100], “The sensor 15 connects with or is separate from the transducer 13. For example, the sensor 15 is a magnetic position sensor with antennas in the transducer 13 as well as antennas spaced from the transducer 13. In another example, the sensor 15 is an optical sensor, such as a camera spaced from the transducer 13. A processor processes images from the optical sensor to determine the position and orientation of the transducer 13.” [0102]);
a processor configured to: scan a region of interest of a subject with an ultrasound beam using the ultrasound probe to acquire B-mode images of a plurality of frames (“The clinician scans the target anatomy in a painting-like motion not being restricted to unidirectional sweeps. To achieve a high-quality 3D reconstruction, the acquired 2D ultrasound images are passed through a processing pipeline recovering orientation and position errors present due to the limited tracking accuracy, and grouping the frames into sequences with uniform orientation. The 2D frames are transformed into 3D space using the orientation and position information, and their image information is compounded into a 3D volume” [0021], “In act 20, data representing a patient is obtained. The data is acquired by scanning the patient. An ultrasound system scans the patient by generating acoustic energy with a transducer and receiving echoes. Any scanning sequence or approach may be used, such as a linear, Vector® or sector scan format of scan lines. The echoes are received by the transducer and converted into electrical signals. The electrical signals are processed to detect intensity (e.g., B-mode), tissue strain, power Doppler, contrast, tissue harmonics, elastic modulus, relaxation time, other tissue property, or other scalar information.” [0030]).
Kapoor does not specifically disclose the region of interest including a wound portion; calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames, the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels
and display a region which has been repeatedly scanned a larger number of times in the three- dimensional probability maps with a darker color or a higher density regardless of the presence probabilities (“an image is generated from the confidence information. Using just the confidence values for each voxel or using the confidence values and ultrasound values, the image is generated and displayed. For using both, the confidence values may be used to modulate a color, brightness or other display characteristic. For using confidence values alone, the confidence values prior to rendering or the resulting pixels after rendering are mapped to display values. As discussed above, a maximum confidence value is selected for each voxel as part of compounding. By displaying an image rendered from the maximum confidence values associated with each pixel and/or voxel, regions of the volume associated with lesser confidence may be identified by the user. The user then moves the transducer to acquire ultrasound values for that region to incrementally compound with the more uncertain values previously obtained. This confidence information is exposed to the clinician in order to assist the clinician in determining regions that need further examination for a complete reconstruction.” [0097]).
However, in a similar field of endeavor, McCormac teaches processing frames of video data and labelling surface elements within a representation of a three-dimensional (3D) space [0002].
McCormac also teaches calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames (“Video data 170 may further comprise any measured sensory input that is arranged in a two-dimensional form representative of a captured or recorded view of a 3D space with a capture device. For example, this may comprise just one of depth data or photometric data, electromagnetic imaging, ultrasonic imaging and radar output, amongst others.” [0061], “At block 605, object-label probability values for spatial elements of frames of video data are determined using a two-dimensional image classifier. For example, the two-dimensional image classifier may be a CNN as described above that is configured to receive a frame of video data as a 2D image and to output probability values for at least one area of the 2D image. In this case, the probability values relate to a plurality of available object or class labels, e.g. string labels representing words and/or identifiers for object data definitions. In one case, probability values may be output for each pixel of an input 2D image and this output may be structured as one or more images, wherein in certain cases each image comprises a probability distribution for a particular object label. These output images can be referred to as “pixel maps”. This may be the case where the two-dimensional image classifier comprises a deconvolutional neural network communicatively coupled to an output of a CNN. The two-dimensional image classifier may be configured to compute object-label probability values based on at least colour data and depth data for a frame.” [0100]);
generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings (“In parallel with the operation of components 515 to 560, the image classifier 565 is configured to receive a frame of video data, Ft, and compute a set of object-label probability distributions 570. In this case, as described above, the object-label probability distributions 570 are provided as a set of images, each image corresponding to a different object label, wherein spatial element values, in this case pixel values, in the images represent a probability that an object having the label is visible in the spatial element. In FIG. 5, the set of object-label probability distributions 570 are received by a surfel-to-pixel mapper 575. In this example, the surfel-to-pixel mapper 575 is configured to determine a correspondence between a spatial element (e.g. a pixel) in the images 570 and a surfel in the surfel representation 530. This correspondence is based on the tracked pose, Pt, which may be stored with the surfel representation 530 or supplied by the frame-to-model tracking component 515. Using the tracked pose, Pt, each surfel at a given 3D location wx(s) in the surfel representation 530 may be associated with pixel coordinates u associated with a frame Ft via the camera projection u(s, t)=π(cwPtwx(s)), where cwPt is the pose of the capture device (“C”) with respect to the “world” (“W”) of the 3D representation which is calculated employing the homogeneous transformation matrix cwPt=wcPt −1 (where wcPt is the pose Pt of the “world” with respect to the capture device) and using homogeneous 3D coordinates.” [0091];
and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels (“In FIG. 5, once a set of object-label probability values 580 have been assigned to surfels deemed visible in a frame of video data they are sent to the Bayesian update engine 585. For example, the surfel-to-pixel mapper 575 may be configured to output a set of surfel-probability pairings, identifying a given surfel (e.g. “SF1234”) and the corresponding probability values from the image classifier 565. The Bayesian update engine 585 is configured to retrieve existing object-label probability values, SPt-1, for surfels identified in the data 580 and to use the new probability values in data 580 to update these existing values. For example, Bayesian update engine 585 may be configured to update all the surfels in a visible set from the surfel representation 530 with a corresponding probability distribution by means of a recursive Bayesian update.” [0092]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with calculate presence probabilities of multiple types of findings related to the wound portion and background region different from the multiple types of findings for each pixel in each of the B-mode images of the plurality of frames, by performing image recognition on the each of the B- mode images of the plurality of frames; generate three-dimensional probability maps for the multiple types of findings based on the positional information of the ultrasound probe acquired by the position sensor and the presence probabilities of the multiple types of findings, each of the three-dimensional probability maps corresponding to each of the multiple types of findings; and generate an integrated probability map by integrating the three-dimensional probability maps, wherein by allotting a presence probability of either one of the background region and each of the multiple types of findings to each of positions represented by the positional information in each of the three- dimensional probability maps, each of the three-dimensional probability maps is composed of a plurality of background voxels to which the presence probability of the background region is allotted and a plurality of finding voxels to which the presence probability of corresponding one of the multiple types of findings is allotted, and by allotting a presence probability of one of the background region and the multiple types of findings to each of the positions represented by the positional information in the integrated probability map, the integrated probability map is composed of the plurality of background voxels and multiple types of finding voxels as taught by McCormac, because there is still a desire for useable methods of processing video data to enable detection and labelling of objects deemed to be present in a scene [0011].
Kapoor in view of McCormac does not specifically teach the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other.
However, in a similar field of endeavor, Aoi teaches using ultrasound to evaluate deep tissue damage under pressure ulcers, and analyzed its potential usefulness in diagnosing deep tissue in jury at the early stages and predicting the prog nosis of pressure ulcers [Pg. 541 Col. 1].
Aoi also teaches the region of interest including a wound portion (“Representative findings from ultrasonic images specific to deep tissue injury in pressure ulcers. Unclear layered structure (ULS) with a coarse echo texture (violet arrow) is most frequently seen in pressure ulcers, whereas a hypoechoic lesion (HEL)(blue arrowhead)is suspected of reflecting a small lesion with fluid, such as seroma and hematoma (above). A superficial fascia overlying the muscles is clearly seen as a hyperechoic line on the ultrasound images(center, right), whereas discontinuous superficial or deep fascia (DCF)(yellow arrowheads) shows a broken or discontinuous line with heterogeneous echo signals (center, left). A heterogeneous hypoechoic area (HHA) is shown by an orange dashed circle (below). The superficial fascia is indicated with a red arrow” [Fig. 1 Pg. 542]);
PNG
media_image1.png
721
812
media_image1.png
Greyscale
the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other (“An unclear layered structure was observed in all 12 patients followed and seems to be the ultrasonic sign most commonly seen in deep tissue injury. An unclear layered structure likely reflects subcutaneous tissue damage such as edema, interstitial fluid, and inflammation that result in a coarse echo texture and the unclear margins of each layer in echoic images.” [Pg. 546 Col. 2], “The image of a heterogeneous hypoechoic area would represent a necrotic area containing necrotic tissue, debris, and fluid; the necrotic area shows a heterogeneous area on ultrasound, with relatively low internal echo.” [Pg. 548 Col. 1]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Kapoor as outlined above with the region of interest including a wound portion; the multiple types of findings including at least one of an edema, a necrosis, and an abscess, the multiple types of findings being different from each other as taught by Aoi, because intermediate-frequency ultrasonography can diagnose deep tissue injury and predict ulcer progression [Pg. 545 Col. 2].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN MALDONADO whose telephone number is 703-756-1421. The examiner can normally be reached 8:00 am-4:00 pm PST M-Th Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571) 272-7230. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Steven Maldonado/
Patent Examiner, Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797
