DETAILED ACTION
Status of Claims
Claims 1-20 are pending in this application, with claims 1, 13 and 17 being independent.
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Obligation Under 37 CFR 1.56 – Joint Inventors
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Drawings
The drawings were received on August 8, 2024. These drawings are acceptable.
Claim Objections
Claims 6-12 are objected to under 37 CFR 1.75(c) as being in improper form because a multiple dependent claim cannot depend from any other multiple dependent claim. See MPEP § 608.01(n). Accordingly, claims 6-12 have not been further treated on the merits.
Claim 1 is objected to because of the following informalities:
“IVUS” (in line 1) is an acronym, or abbreviation. The meaning of the acronym in the claim is undefined. Appropriate correction is required.
“the computing device” (in line 2) lacks proper antecedent basis. Amending to instead recite “a computing device” will cure the deficiency. Appropriate correction is required.
Claim 17 is objected to because of the following informalities:
“an intravascular imaging device” (line 6 of claim 17) lacks proper antecedent basis. Amending to instead recite “the intravascular imaging device” will correct the deficiency. Appropriate correction is required.
Claim 18 is objected to because of the following informalities:
“the apparatus” (line 2 of claim 18) lacks proper antecedent basis. Amending to replace “the apparatus” with “the processor” will correct the deficiency. Appropriate correction is required.
Claim 19 is objected to because of the following informalities:
line 1 of claim 19 recites “The computer-readable storage device of claim 17,” however, claim 17 is directed to “An apparatus”. Amending claim 19 to instead recite “The apparatus of claim 17,” would fix the problem. Appropriate correction is required.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art;
Ascertaining the differences between the prior art and the claims at issue;
Resolving the level of ordinary skill in the pertinent art; and
Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 13-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over STEINBERG et al. (US 2014/0094691, hereinafter “STEINBERG”) in view of RAJGURU et al. (US 2023/0190224, hereinafter referred to as “RAJGURU”).
Regarding claim 17, STEINBERG discloses an apparatus (e.g., FIG. 1B) comprising:
a processor (e.g., ¶ [0320]: “processor 20.”) arranged to be coupled to an intravascular imaging device (e.g., ¶ [0320]: “an endoluminal data-acquisition device 16,” ¶ [0434]: “an IVUS probe” ¶ [0469]: “data acquired by a first endoluminal modality (e.g., IVUS)”) and a fluoroscope device (e.g., ¶ [0320]: “an extraluminal image acquisition device 17,” ¶ [0315]: “the extraluminal imaging is fluoroscopy,” ¶ [0411]: “fluoroscopic images of the data-acquisition device within the lumen are acquired” ¶ [0412]: “Typically, image processing of fluoroscopic images of the data-acquisition device within the lumen can be used to identify forward motion of the data-acquisition device” ¶ [0456]: “For some applications, the IVUS images are overlaid on the fluoroscopic images. For some applications, the IVUS images are fused with the fluoroscopic images.” ¶ [0475]: “extraluminal imaging (e.g., fluoroscopic imaging),” ¶ [0469]: “data acquired by a first endoluminal modality (e.g., IVUS) are co-registered with the fluoroscopic image stream,” ¶ [0478]: “co-using extraluminal images and endoluminal data are described hereinabove primarily with respect to extraluminal fluoroscopic/angiographic images and endoluminal IVUS images,”) (¶ [0320]: “Reference is also made to FIG. 1B, which is a block diagram of an endoluminal data-acquisition device 16, an extraluminal image acquisition device 17, a user interface 18, a display 19, and a processor 20.”); and
a memory storage device coupled to the processor, the memory storage device comprising instructions, which when executed by the processor cause the apparatus (¶ [0320]: “Processor 20 typically includes at least some of the following functionalities, the functions of which are described in further detail hereinbelow: roadmap-image-designation functionality 21, pathway-designation functionality 22, landmark-classification functionality 23, feature-identifying functionality 24, roadmap-mapping functionality 25, location-interpolation functionality 26, pathway-calibration functionality 27, co-registration functionality 28, stack-generation functionality 29, parameter-measurement functionality 30, duplicate-data-point-identification functionality 31, data-point-selection functionality 32, display-driving functionality 33, direction-determination functionality 34, output-generation functionality 35, and/or region-identification functionality 36.” ¶ [0373]: “a single computer (or two or more computers that are time-synchronized) may operate both the extraluminal imaging and the endoluminal data-acquisition,” NOTE: One of ordinary skill in the art would understand that the described process(es) performed using a computer including processor 20 would, by necessity, require a memory storage device coupled to the processor storing instructions for execution by the processor to perform the disclosed processing.) to:
receive, from an intravascular imaging device (¶ [0320]: “an endoluminal data-acquisition device 16,” ¶ [0434]: “an IVUS probe”), a plurality of images associated with a vessel of a patient (¶ [0434]: “endoluminal data points”; ¶ [0453]: “data acquired by a first endoluminal modality (e.g., IVUS)”; ¶ [0455]: “an IVUS stack comprising data from IVUS images” ¶ [0478]: “endoluminal IVUS images”), the plurality of images comprising multidimensional (e.g., As shown in FIG. 6, endoluminal image 66 is two dimensional with a third dimension being the time and/or its location along the vessel when the image data was captured.) and multivariate images (e.g., ¶ [0327]: “a plurality of features that are typically visible in extraluminal images of the lumen that are acquired during the movement of the endoluminal data-acquisition device through the lumen,”) (NOTE: While STEINBERG may not explicitly disclose that the plurality of received IVUS images (i.e., frames) comprise multidimensional and multivariate images, in ¶ [0034] of applicant’s Specification, applicant admits that IVUS images are “multidimensional and multivariate images.” Thus, STEINBERG inherently discloses that the images are multidimensional and multivariate images. See paragraph ¶ [0034]: “In general, IVUS image frames 122 are multi-dimensional multivariate images”) (¶ [0332]: “It is noted that, in general, the scope of the present application includes performing the techniques described herein with an endoluminal data-acquisition device that acquires data points while the data-acquisition device is being advanced distally through the lumen, and/or an endoluminal data-acquisition device that acquires data points while the data-acquisition device is being retracted proximally through the lumen. It is further noted that, in general, the scope of the present application includes performing the techniques described herein with an endoluminal data-acquisition device that acquires images of the lumen and/or a data-acquisition device that acquires functional data regarding the lumen.” ¶ [0333]: “Typically, data are acquired at and/or in the vicinity of the designated site. Typically, a plurality of data points (e.g., images) are acquired at respective locations along the lumen. It is noted that, for some applications, data are acquired subsequent to the initial insertion of the data-acquisition device into the lumen. For example, when data are acquired from blood vessels, the data-acquisition device is typically inserted into the blood vessel to beyond the site of interest under extraluminal imaging (e.g., fluoroscopy), and data acquisition is performed during (manual or automated) pullback of the data-acquisition device through the blood vessel.” ¶ [0478]: “Examples of the anatomical structure to which the aforementioned co-registration of extraluminal and endoluminal images may be applied include a coronary vessel, a coronary lesion, a vessel, a vascular lesion, a lumen, a luminal lesion, and/or a valve.”);
generate, for each of the plurality of images (e.g., ¶ [0437]: “of each of the image frames.” ¶ [0437]: “identify a region of one of the endoluminal images as having a given characteristic” and ¶ [0437]: “search for a region in an adjacent endoluminal image that has the same characteristic,”), a mask (¶ [0437]: “identify a region of one of the endoluminal images as having a given characteristic” ¶ [0437]: “a region in an adjacent endoluminal image that has the same characteristic,”) comprising indications of key features of the vessel (e.g., ¶ [0437]: “identify a region of one of the endoluminal images as having a given characteristic” ¶ [0437]: “a region in an adjacent endoluminal image that has the same characteristic,” ¶ [0437]: “e.g., being lighter than the surrounding regions”) (¶ [0437]: “endoluminal data points (e.g., images) are aligned with each other in the stack, using image processing techniques. For example, stack-generation functionality 29 may identify a region of one of the endoluminal images as having a given characteristic (e.g., being lighter than the surrounding regions). Stack-generation functionality 29 may then search for a region in an adjacent endoluminal image that has the same characteristic, and may align the adjacent image frames by aligning the regions of each of the image frames.”); and
align the plurality of images (e.g., ¶ [0437]: “align the adjacent image frames by aligning the regions of each of the image frames.” ¶ [0435]: “generating a stack of endoluminal data points (e.g., endoluminal images) in which non-uniform longitudinal motion of a portion of the endoluminal data-acquisition device is accounted for,”) based on the plurality of masks (e.g., ¶ [0437]: “align the adjacent image frames by aligning the regions of each of the image frames.” ¶ [0437]: “a region of one of the endoluminal images as having a given characteristic” ¶ [0437]: “a region in an adjacent endoluminal image that has the same characteristic,”) (¶ [0434]: “Typically, while an endoluminal data-acquisition device is moved through a lumen (e.g., while an IVUS probe is pulled back or pushed forward through a blood vessel), the device undergoes non-longitudinal motion. For example, the data-acquiring portion of the device (e.g., the head of the device) typically moves in an axial direction, rotates about the longitudinal axis of the device, and/or becomes tilted. For some applications, stack-generation functionality 29 determines that endoluminal data points are not aligned with each other due to non-longitudinal motion undergone by a portion of the endoluminal data-acquisition device with respect to the lumen, between acquisitions of respective endoluminal data points. In response thereto, stack-generation functionality 29 aligns the endoluminal data points with each other, to account for the non-longitudinal motion undergone by the portion of the endoluminal data-acquisition device.” ¶ [0437]: “endoluminal data points (e.g., images) are aligned with each other in the stack, using image processing techniques. For example, stack-generation functionality 29 may identify a region of one of the endoluminal images as having a given characteristic (e.g., being lighter than the surrounding regions). Stack-generation functionality 29 may then search for a region in an adjacent endoluminal image that has the same characteristic, and may align the adjacent image frames by aligning the regions of each of the image frames.” ¶ [0435]: “generating a stack of endoluminal data points (e.g., endoluminal images) in which non-uniform longitudinal motion of a portion of the endoluminal data-acquisition device is accounted for,” ¶ [0456]: “a three-dimensional "tunnel-like" reconstruction of the IVUS images of the vessel”).
STEINBERG does not unequivocally disclose: “a mask comprising indications of key features of the vessel.”
However, whereas STEINBERG may not be entirely explicit as to, RAJGURU clearly teaches an apparatus (e.g., ¶ [0031]: “an intraluminal imaging and x-ray system 100,” See FIG. 1.) comprising:
a processor (e.g., ¶ [0035]: “processor 134,”) arranged to be coupled to an intravascular imaging device (e.g., ¶ [0033]: “an intraluminal sensing system 101” ¶ [0034]: “The intraluminal imaging system 101 can be an ultrasound imaging system. In some instances, the intraluminal imaging system 101 can be an intravascular ultrasound (IVUS) imaging system. The intraluminal imaging system 101 may include an intraluminal imaging device 102,” ¶ [0034]: “The intraluminal imaging device 102 can be an ultrasound imaging device. In some instances, the device 102 can be an IVUS imaging device, such as a solid-state IVUS device.” ¶ [0037]: “The lumen 120 may be a blood vessel, such as an artery or a vein of a patient's vascular system, including cardiac vasculature, peripheral vasculature, neural vasculature, renal vasculature, and/or any other suitable lumen inside the body.”) and a fluoroscope device (e.g., ¶ [0033]: “and an extraluminal imaging system 151.” ¶ [0042]: “The x-ray imaging system 151 may include an x-ray imaging apparatus or device 152 configured to perform x-ray imaging, angiography, fluoroscopy, radiography, venography, among other imaging techniques.”) (¶ [0032]: “as shown in FIG. 1, the intraluminal imaging system 101 and the x-ray imaging system 151 may be in communication with the same control system 130. In this embodiment, both systems may be in communication with the same display 132, processor 134, and communication interface 140 shown as well as in communication with any other components implemented within the control system 130.” ¶ [0034]: “The intraluminal imaging system 101 may include an intraluminal imaging device 102, such as a catheter, guide wire, or guide catheter, in communication with the control system 130.” ¶ [0042]: “The x-ray imaging device 152 may also be in communication with the control system 130.” ¶ [0051]: “The processor 134 receives the x-ray data from the x-ray device 152 by way of the communication interface 140” ¶ [0071]: “the processor circuit 510 may be in communication with intraluminal imaging device 102, the x-ray imaging device 152, the display 132 within the system 100. The processor circuit 510 may include the processor 134 and/or the communication interface 140 (FIG. 1). One or more processor circuits 510 are configured to execute the operations described herein. As shown, the processor circuit 510 may include a processor 560, a memory 564, and a communication module 568. These elements may be in direct or indirect communication with each other, for example via one or more buses.”); and
a memory storage device coupled to the processor, the memory storage device comprising instructions, which when executed by the processor cause the apparatus (¶ [0035]: “The control system 130, including the processor 134, can be operable to facilitate the features of the IVUS imaging system 101 described herein. For example, the processor 134 can execute computer readable instructions stored on the non-transitory tangible computer readable medium.” ¶ [0071]: “the processor circuit 510 may be in communication with intraluminal imaging device 102, the x-ray imaging device 152, the display 132 within the system 100. The processor circuit 510 may include the processor 134 and/or the communication interface 140 (FIG. 1). One or more processor circuits 510 are configured to execute the operations described herein. As shown, the processor circuit 510 may include a processor 560, a memory 564, and a communication module 568. These elements may be in direct or indirect communication with each other, for example via one or more buses.” ¶ [0073]: “The memory 564 may include a cache memory (e.g., a cache memory of the processor 560), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, other forms of volatile and non-volatile memory, or a combination of different types of memory. In an embodiment, the memory 564 includes a non-transitory computer-readable medium. The memory 564 may store instructions 566. The instructions 566 may include instructions that, when executed by the processor 560, cause the processor 560 to perform the operations described herein with reference to the probe 110 and/or the host 130 (FIG. 1).”) to:
receive, from an intravascular imaging device (e.g., ¶ [0158]: “the device 102”), a plurality of images associated with a vessel of a patient (e.g., ¶ [0158]: “the ILD 1500 may assist a user of the system 100 to identify where along the imaged vessel an IVUS image was obtained. The ILD 1500 may be generated by the system 100 based on the IVUS images received by the device 102 (FIG. 1). In some applications, the ILD 1500 may provide a longitudinal cross-sectional view of the blood vessel. In contrast, an IVUS image is a tomographic or radial cross-sectional view of the blood vessel. The ILD 1500 can be a stack of the intraluminal images acquired at various positions along the vessel, such that the longitudinal view of the ILD 1500 is perpendicular to the radial cross-sectional view of the intraluminal image. In such an embodiment, the ILD 1500 may show the length of the vessel, whereas an individual intraluminal image is a single radial cross-sectional image at a given location along the length.” ¶ [0181]: “Referring now to FIG. 17, at step 1710, the method 1700 includes receiving intravascular data from an intravascular catheter. For example, the processor circuit may receive IVUS imaging data from an IVUS imaging catheter.”), the plurality of images comprising multidimensional and multivariate images (¶ [0075]: “The ultrasound imaging device 602 may be similar to the imaging device 102 described with reference to FIGS. 1-4. While the imaging device may be of any suitable type or modality, in some embodiments, the imaging device 602 may acquire intravascular images and/or other data via ultrasound transmission. In such an embodiment, the imaging device 602 may capture data corresponding to the surface of the lumen including the inner surface of the vessel wall or the inner surface of various obstructions within the vessel, as well as data corresponding to structures beneath the lumen surface such as the wall of the vessel, anatomies outside the vessel, the contents of various obstructions within the vessel or any other data.” ¶ [0124]: “two-dimensional and/or cross-sectional region of an IVUS image”); and
generate, for each of the plurality of images (e.g., ¶ [0075]: “the imaging device 602 may acquire intravascular images and/or other data via ultrasound transmission. In such an embodiment, the imaging device 602 may capture data corresponding to the surface of the lumen including the inner surface of the vessel wall or the inner surface of various obstructions within the vessel, as well as data corresponding to structures beneath the lumen surface such as the wall of the vessel, anatomies outside the vessel, the contents of various obstructions within the vessel or any other data.” ¶ [0084]: “The input images 710 can be an image stream of frames continuously acquired (e.g., in real time) by an imaging probe during a pullback procedure. The input images 710 can be transmitted from the ultrasound probe 102 (FIG. 1) to the analysis module 720 as the images are being acquired or after the images are acquired.”), a mask (e.g., ¶ [0076]: “A region 612” ¶ [0165]: “an overlay 1630”) comprising indications of key features of the vessel (¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present.” ¶ [0076]: “the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium. ¶ [0165]: “the graphical overlay 1630 provides a visual representation of the calcium 1622.”) (¶ [0075]: “The ultrasound imaging device 602 may be similar to the imaging device 102 described with reference to FIGS. 1-4. While the imaging device may be of any suitable type or modality, in some embodiments, the imaging device 602 may acquire intravascular images and/or other data via ultrasound transmission. In such an embodiment, the imaging device 602 may capture data corresponding to the surface of the lumen including the inner surface of the vessel wall or the inner surface of various obstructions within the vessel, as well as data corresponding to structures beneath the lumen surface such as the wall of the vessel, anatomies outside the vessel, the contents of various obstructions within the vessel or any other data.” ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present. This obstruction 610 may restrict blood flow causing a variety of unpleasant or potentially dangerous conditions for the patient. As shown in the image 600, it may be difficult for a user of the system 100, particularly an inexperienced user, to determine what kind of obstruction is shown in an ultrasound image and to what extent the obstruction has grown or constricted blood flow. For example, an obstruction may include plaque, such as calcified deposits (e.g. calcium), dense calcium, fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various blood cells, dead blood cells, muscle cells, or other materials. An obstruction may include any or all of these materials in various compositions. The material of the obstruction and the composition of the obstruction may influence both the severity of the condition and immediacy of any need for treatment as well as the type of treatment recommended. The disclosed systems, devices, and methods advantageously identify the composition of various materials within obstructions in a vessel and succinctly convey this information to the user for enhanced diagnosis and treatment, as will be described in more detail hereafter. The obstruction may be referred to as a tissue. The tissue may include multiple tissue types other than calcium and the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium. The other tissue types (e.g., 1624 of FIGS. 16 and/or 1022 of FIG. 10) are grouped together such that the graphical overlay 1630 only includes a visual representation of the calcium and the visual representation of the multiple tissue types other than calcium.” ¶ [0100]: “In some embodiments, various outputs 730 of the machine learning algorithm 700 may be determined based on other outputs 730. For example, in some embodiments, the calcium depth 733, calcium thickness 734, calcium arc 735, and calcium distribution 736 may be determined based on the identified calcium and plaque locations 731 and vessel and lumen borders 732. For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques such as [list image processing techniques here]. In other embodiments, the processor circuit may employ a deep learning algorithm to generate the outputs 733-737. The deep learning algorithm used may be the same algorithm as the machine learning algorithm 700 described with reference to FIG. 7, the convolutional neural network 800 described with reference to FIG. 8, or may be a separate deep learning algorithm.” ¶ [0165]: “Such an overlay 1630 is shown overlaid over the IVUS image 1640 in FIG. 16. The screen display (e.g., the graphical user interface 1600) includes a graphical overlay 1630 overlaid on the IVUS image 1640 and the graphical overlay 1630 provides a visual representation of the calcium 1622.” ¶ [0181]: “At step 1720, the method 1700 includes identifying calcium based on the intravascular data using a deep learning network. For example, the processor circuit may identify calcium based on the IVUS imaging data. At step 1730, the method 1700 includes determining, based on the identification of calcium, a distance between the calcium and the lumen border, and/or a density of the calcium relative to tissue.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG and RAJGURU, it would have been obvious to one of ordinary skill in the art to have modified the apparatus taught by STEINBERG so as to include generating, for each of the plurality of images, a mask comprising indications of key features of the vessel, as taught by RAJGURU.
Regarding claim 20 (depends on claim 17), STEINBERG discloses:
wherein the intravascular imaging device is an intravascular ultrasound (IVUS) probe (e.g., ¶ [0320]: “an endoluminal data-acquisition device 16,” ¶ [0434]: “an IVUS probe” ¶ [0469]: “data acquired by a first endoluminal modality (e.g., IVUS)”).
Regarding claim 1, claim 1 corresponds to the method implemented by the apparatus of claim 20 (i.e., claims 17 and 20) and, as such, is rejected for the same reasons applied above in the rejection of claim 20 (i.e., claims 17 and 20).
Regarding claim 2 (depends on claim 1), whereas STEINBERG may not be entirely explicit as to, RAJGURU further teaches:
wherein the key features comprise at least one of a vessel border (¶ [0078]: “vessel borders and lumen borders 732,”), a lumen border (¶ [0078]: “vessel borders and lumen borders 732,”), plaque (¶ [0078]: “calcium plaque locations 731,” ¶ [0095]: “Calcium deposits 920 and/or other plaque 922”), or a lesion (¶ [0175]: “a lesion or an obstruction.”).
Regarding claim 3 (depends on claim 2), whereas STEINBERG may not be entirely explicit as to, and RAJGURU further teaches:
detecting, by the computing device (¶ [0077]: “performed by the processor 134”), the vessel border and the lumen border (e.g., ¶ [0078]: “vessel borders and lumen borders 732,”) (¶ [0077]: “the artificial intelligence framework 700 may include a machine learning network and various aspects of the framework may be performed by the processor 134 and/or the processor circuit 510 and may include instructions similar to the instructions 566 previously described. The machine learning algorithm 700 may also be a deep learning algorithm in some embodiments. The processor circuit 510 may be configured to use a machine learning network to identify the calcium.” ¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0088]: “FIG. 8 is a schematic diagram of a convolutional neural network (CNN) configuration 800, according to aspects of the present disclosure. For example, the CNN configuration 800 can be implemented as the deep learning network 724 (FIG. 7). In an embodiment, the configuration 800 may perform a classification task. For example, a convolutional neural network (CNN) may provide classification labels for each pixel of an IVUS image or signal.” ¶ [0088]: “The configuration 800 can be trained for identification of calcium deposits, other plaque, vessel borders, and/or lumen borders associated with received ultrasound images as described in greater detail below.”);
inferring, by the computing device using a machine learning (ML) model (e.g., ¶ [0077]: “machine learning network and various aspects of the framework may be performed by the processor 134”; ¶ [0078]: “machine learning algorithm 700”), the plaque (¶ [0078]: “calcium plaque locations 731,” ¶ [0095]: “calcium deposits 920” ) or the lesion (¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.”); and
generating the mask (e.g., ¶ [0078]: “generate a visual representation of calcium location 740.” ¶ [0094]: “an overlay,” … ¶ [0094]: “to identify calcium deposits 920, other plaque 922, a vessel border 930, and a lumen border 932.” ¶ [0165]: “overlay 1630”) comprising indications of the detected vessel border (¶ [0094]: “vessel border 930,”), the detected lumen border (¶ [0094]: “lumen border 932.”), and the inferred plaque or lesion (¶ [0078]: “generate a visual representation of calcium location 740.” ¶ [0094]: “calcium deposits 920, other plaque 922,”) (¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0090]: “In that regard, the CNN 800 can be a multi-class classification network. In an exemplary embodiment, the plurality of classes 832 may include the location of calcium deposits within an intravascular image, the location of other plaque within the image, a vessel border in the image, or a lumen border in the image. The output 830 may indicate how likely various regions of the input image 805 belongs to or corresponds to a particular class 832.” ¶ [0092]: “In some embodiments, multiple convolutional neural networks may be implemented to identify different characteristics of received ultrasound images. For example, one CNN may be trained to identify calcium plaque within a received ultrasound image, a separate CNN may be trained to identify other plaque, and additional CNN's may identify the vessel border and lumen border respectively. Any CNN may be trained to identify any one of these characteristics within a received ultrasound image, including just one, some, or all of these characteristics.” ¶ [0094]: “FIG. 9 is a diagrammatic view of an intravascular ultrasound image 600 with an overlay, according to aspects of the present disclosure. FIG. 9 may represent the view of the ultrasound image 600 after the processor circuit 510 has completed implementation of the machine learning algorithm 700 (FIG. 7) and/or the convolutional neural network configuration (FIG. 8) to identify calcium deposits 920, other plaque 922, a vessel border 930, and a lumen border 932. FIG. 9 will be described with reference to FIG. 7.” ¶ [0100]: “For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques” ¶ [0165]: “Such an overlay 1630 is shown overlaid over the IVUS image 1640 in FIG. 16. The screen display (e.g., the graphical user interface 1600) includes a graphical overlay 1630 overlaid on the IVUS image 1640 and the graphical overlay 1630 provides a visual representation of the calcium 1622.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG and RAJGURU, it would have been obvious to one of ordinary skill in the art to have modified the method taught by STEINBERG so as to include detecting the vessel border and the lumen border, inferring, using the machine learning model, the plaque or lesion, and generating the mask comprising indications of key features of the vessel comprising at least one of a vessel border, a lumen border, plaque or a lesion, as taught by RAJGURU.
Regarding claim 4 (depends on claim 2), whereas STEINBERG may not be entirely explicit as to, RAJGURU further teaches:
inferring, by the computing device (e.g., ¶ [0035]: “processor 134,” ¶ [0086]: “The deep learning network 724 may receive as an input the processed images 710 from the preprocessor 722. The deep learning network 724 can include hardware (e.g., electrical circuit components) and/or software algorithms (e.g., executed by a processor). The deep learning network 724 may then identify various parameters associated with the received ultrasound images 710.”) using a machine learning (ML) model (e.g., ¶ [0078]: “machine learning algorithm 700”), the plurality of masks (e.g., ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present.” ¶ [0076]: “the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium.” ¶ [0165]: “the graphical overlay 1630 provides a visual representation of the calcium 1622.” ¶ [0117]: “visual representation of calcium location 740 may be direct outputs of the machine learning algorithm 700”) from the plurality of images (e.g., ¶ [0075]: “The ultrasound imaging device 602 may be similar to the imaging device 102 described with reference to FIGS. 1-4. While the imaging device may be of any suitable type or modality, in some embodiments, the imaging device 602 may acquire intravascular images and/or other data via ultrasound transmission. In such an embodiment, the imaging device 602 may capture data corresponding to the surface of the lumen including the inner surface of the vessel wall or the inner surface of various obstructions within the vessel, as well as data corresponding to structures beneath the lumen surface such as the wall of the vessel, anatomies outside the vessel, the contents of various obstructions within the vessel or any other data.” ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present. This obstruction 610 may restrict blood flow causing a variety of unpleasant or potentially dangerous conditions for the patient. As shown in the image 600, it may be difficult for a user of the system 100, particularly an inexperienced user, to determine what kind of obstruction is shown in an ultrasound image and to what extent the obstruction has grown or constricted blood flow. For example, an obstruction may include plaque, such as calcified deposits (e.g. calcium), dense calcium, fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various blood cells, dead blood cells, muscle cells, or other materials. An obstruction may include any or all of these materials in various compositions. The material of the obstruction and the composition of the obstruction may influence both the severity of the condition and immediacy of any need for treatment as well as the type of treatment recommended. The disclosed systems, devices, and methods advantageously identify the composition of various materials within obstructions in a vessel and succinctly convey this information to the user for enhanced diagnosis and treatment, as will be described in more detail hereafter. The obstruction may be referred to as a tissue. The tissue may include multiple tissue types other than calcium and the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium. The other tissue types (e.g., 1624 of FIGS. 16 and/or 1022 of FIG. 10) are grouped together such that the graphical overlay 1630 only includes a visual representation of the calcium and the visual representation of the multiple tissue types other than calcium.” ¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0084]: “The input images 710 can be an image stream of frames continuously acquired (e.g., in real time) by an imaging probe during a pullback procedure. The input images 710 can be transmitted from the ultrasound probe 102 (FIG. 1) to the analysis module 720 as the images are being acquired or after the images are acquired.” ¶ [0095]: “As shown in FIG. 7, the deep learning network may be trained to identify calcium and plaque locations 731 within a received intravascular image 710. Similarly, as shown in FIG. 9, the system 100 may identify regions within the image 600 where calcium deposits 920 and other plaque 922 is present. Calcium deposits 920 and/or other plaque 922 may be observed at various locations within a vessel and may be part of other structures. As shown in FIG. 9, a large region of other plaque 922 is shown completely lining the vessel wall 930. As described, this other plaque 922 may include fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various dead blood cells or other tissues, or other materials which may build up over time.” ¶ [0096]: “The vessel border 930 and lumen border 932 are additionally identified by the deep learning network as shown in FIG. 9.” ¶ [0096]: “The machine learning algorithm 700 may be trained to identify the vessel border and lumen border 732.” ¶ [0100]: “In some embodiments, various outputs 730 of the machine learning algorithm 700 may be determined based on other outputs 730. For example, in some embodiments, the calcium depth 733, calcium thickness 734, calcium arc 735, and calcium distribution 736 may be determined based on the identified calcium and plaque locations 731 and vessel and lumen borders 732. For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques such as [list image processing techniques here]. In other embodiments, the processor circuit may employ a deep learning algorithm to generate the outputs 733-737. The deep learning algorithm used may be the same algorithm as the machine learning algorithm 700 described with reference to FIG. 7, the convolutional neural network 800 described with reference to FIG. 8, or may be a separate deep learning algorithm.” ¶ [0101]: “As shown in FIG. 10, locations within the image 1000 corresponding to calcium deposits 1020, other plaque 1022, the vessel border 1030, and the lumen border 1032 are identified. These features may be identified according to principles similar to those described with reference to the deep learning network algorithm of FIG. 7, the convolutional neural network of FIG. 8, or methods described with reference to FIG. 9.” ¶ [0117]: “As shown in FIG. 7, additional outputs 730 of the machine learning algorithm 700 may include scan line scores 739 and a visual representation of calcium location 740. The scan line scores 739 and visual representation of calcium location 740 may be direct outputs of the machine learning algorithm 700 and independent of any other outputs 730. However, in some embodiments, the scan lines scores 739 and visual representation of calcium location 740 may be determined by the processor circuit 510 (FIG. 5) based on various outputs 730, such as the calcium and other plaque locations 731, the vessel border and lumen border 732, calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and/or calcium length 737.” ¶ [0181]: “Referring now to FIG. 17, at step 1710, the method 1700 includes receiving intravascular data from an intravascular catheter. For example, the processor circuit may receive IVUS imaging data from an IVUS imaging catheter. At step 1720, the method 1700 includes . For example, the processor circuit may identify calcium based on the IVUS imaging data.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG and RAJGURU, it would have been obvious to one of ordinary skill in the art to have modified the method taught by STEINBERG so as to include using a machine learning model to infer the plurality of masks from the plurality of images, as taught by RAJGURU.
Regarding claim 13, claim 13 is directed to the memory storage device in the apparatus of claim 17, and, as such, claim 13 is rejected for the same reasons applied above in the rejection of claim 17.
Regarding claim 14 (depends on claim 13), whereas STEINBERG may not be entirely explicit as to, and RAJGURU further teaches:
wherein the key features comprise at least one of a vessel border (¶ [0078]: “vessel borders and lumen borders 732,”), a lumen border (¶ [0078]: “vessel borders and lumen borders 732,”), plaque (¶ [0078]: “calcium plaque locations 731,” ¶ [0095]: “Calcium deposits 920 and/or other plaque 922”), or a lesion (¶ [0175]: “a lesion or an obstruction.”).
Regarding claim 15 (depends on claim 14), whereas STEINBERG may not be entirely explicit as to, and RAJGURU further teaches that the instructions when executed by the processor further cause the computing device to:
detect, by the computing device (¶ [0077]: “performed by the processor 134”), the vessel border and the lumen border (e.g., ¶ [0078]: “vessel borders and lumen borders 732,”) (¶ [0077]: “the artificial intelligence framework 700 may include a machine learning network and various aspects of the framework may be performed by the processor 134 and/or the processor circuit 510 and may include instructions similar to the instructions 566 previously described. The machine learning algorithm 700 may also be a deep learning algorithm in some embodiments. The processor circuit 510 may be configured to use a machine learning network to identify the calcium.” ¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0088]: “FIG. 8 is a schematic diagram of a convolutional neural network (CNN) configuration 800, according to aspects of the present disclosure. For example, the CNN configuration 800 can be implemented as the deep learning network 724 (FIG. 7). In an embodiment, the configuration 800 may perform a classification task. For example, a convolutional neural network (CNN) may provide classification labels for each pixel of an IVUS image or signal.” ¶ [0088]: “The configuration 800 can be trained for identification of calcium deposits, other plaque, vessel borders, and/or lumen borders associated with received ultrasound images as described in greater detail below.”);
infer, by the computing device using a machine learning (ML) model (e.g., ¶ [0077]: “machine learning network and various aspects of the framework may be performed by the processor 134”; ¶ [0078]: “machine learning algorithm 700”), the plaque (¶ [0078]: “calcium plaque locations 731,” ¶ [0095]: “calcium deposits 920” ) or the lesion (¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.”); and
generate the mask (e.g., ¶ [0078]: “generate a visual representation of calcium location 740.” ¶ [0094]: “an overlay,” … ¶ [0094]: “to identify calcium deposits 920, other plaque 922, a vessel border 930, and a lumen border 932.” ¶ [0165]: “overlay 1630”) comprising indications of the detected vessel border (¶ [0094]: “vessel border 930,”), the detected lumen border (¶ [0094]: “lumen border 932.”), and the inferred plaque or lesion (¶ [0078]: “generate a visual representation of calcium location 740.” ¶ [0094]: “calcium deposits 920, other plaque 922,”) (¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0090]: “In that regard, the CNN 800 can be a multi-class classification network. In an exemplary embodiment, the plurality of classes 832 may include the location of calcium deposits within an intravascular image, the location of other plaque within the image, a vessel border in the image, or a lumen border in the image. The output 830 may indicate how likely various regions of the input image 805 belongs to or corresponds to a particular class 832.” ¶ [0092]: “In some embodiments, multiple convolutional neural networks may be implemented to identify different characteristics of received ultrasound images. For example, one CNN may be trained to identify calcium plaque within a received ultrasound image, a separate CNN may be trained to identify other plaque, and additional CNN's may identify the vessel border and lumen border respectively. Any CNN may be trained to identify any one of these characteristics within a received ultrasound image, including just one, some, or all of these characteristics.” ¶ [0094]: “FIG. 9 is a diagrammatic view of an intravascular ultrasound image 600 with an overlay, according to aspects of the present disclosure. FIG. 9 may represent the view of the ultrasound image 600 after the processor circuit 510 has completed implementation of the machine learning algorithm 700 (FIG. 7) and/or the convolutional neural network configuration (FIG. 8) to identify calcium deposits 920, other plaque 922, a vessel border 930, and a lumen border 932. FIG. 9 will be described with reference to FIG. 7.” ¶ [0100]: “For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques” ¶ [0165]: “Such an overlay 1630 is shown overlaid over the IVUS image 1640 in FIG. 16. The screen display (e.g., the graphical user interface 1600) includes a graphical overlay 1630 overlaid on the IVUS image 1640 and the graphical overlay 1630 provides a visual representation of the calcium 1622.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG and RAJGURU, it would have been obvious to one of ordinary skill in the art to have modified the instructions in the computer-readable storage device taught by STEINBERG so as to also include instructions for detecting the vessel border and the lumen border, using the machine learning model to infer the plaque or lesion, and generating the mask comprising indications of key features of the vessel comprising at least one of a vessel border, a lumen border, plaque or a lesion, as taught by RAJGURU.
Regarding claim 16 (depends on claim 14), whereas STEINBERG may not be entirely explicit as to, RAJGURU further teaches that the instructions when executed by the processor further cause the computing device to:
infer, by the computing device (e.g., ¶ [0035]: “processor 134,” ¶ [0086]: “The deep learning network 724 may receive as an input the processed images 710 from the preprocessor 722. The deep learning network 724 can include hardware (e.g., electrical circuit components) and/or software algorithms (e.g., executed by a processor). The deep learning network 724 may then identify various parameters associated with the received ultrasound images 710.”) using a machine learning (ML) model (e.g., ¶ [0078]: “machine learning algorithm 700”), the plurality of masks (e.g., ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present.” ¶ [0076]: “the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium.” ¶ [0165]: “the graphical overlay 1630 provides a visual representation of the calcium 1622.” ¶ [0117]: “visual representation of calcium location 740 may be direct outputs of the machine learning algorithm 700”) from the plurality of images (e.g., ¶ [0075]: “The ultrasound imaging device 602 may be similar to the imaging device 102 described with reference to FIGS. 1-4. While the imaging device may be of any suitable type or modality, in some embodiments, the imaging device 602 may acquire intravascular images and/or other data via ultrasound transmission. In such an embodiment, the imaging device 602 may capture data corresponding to the surface of the lumen including the inner surface of the vessel wall or the inner surface of various obstructions within the vessel, as well as data corresponding to structures beneath the lumen surface such as the wall of the vessel, anatomies outside the vessel, the contents of various obstructions within the vessel or any other data.” ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present. This obstruction 610 may restrict blood flow causing a variety of unpleasant or potentially dangerous conditions for the patient. As shown in the image 600, it may be difficult for a user of the system 100, particularly an inexperienced user, to determine what kind of obstruction is shown in an ultrasound image and to what extent the obstruction has grown or constricted blood flow. For example, an obstruction may include plaque, such as calcified deposits (e.g. calcium), dense calcium, fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various blood cells, dead blood cells, muscle cells, or other materials. An obstruction may include any or all of these materials in various compositions. The material of the obstruction and the composition of the obstruction may influence both the severity of the condition and immediacy of any need for treatment as well as the type of treatment recommended. The disclosed systems, devices, and methods advantageously identify the composition of various materials within obstructions in a vessel and succinctly convey this information to the user for enhanced diagnosis and treatment, as will be described in more detail hereafter. The obstruction may be referred to as a tissue. The tissue may include multiple tissue types other than calcium and the graphical overlay 1630 (FIG. 16) provides a visual representation 1624 (FIG. 16) of the multiple tissue types other than calcium. The other tissue types (e.g., 1624 of FIGS. 16 and/or 1022 of FIG. 10) are grouped together such that the graphical overlay 1630 only includes a visual representation of the calcium and the visual representation of the multiple tissue types other than calcium.” ¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740.” ¶ [0084]: “The input images 710 can be an image stream of frames continuously acquired (e.g., in real time) by an imaging probe during a pullback procedure. The input images 710 can be transmitted from the ultrasound probe 102 (FIG. 1) to the analysis module 720 as the images are being acquired or after the images are acquired.” ¶ [0095]: “As shown in FIG. 7, the deep learning network may be trained to identify calcium and plaque locations 731 within a received intravascular image 710. Similarly, as shown in FIG. 9, the system 100 may identify regions within the image 600 where calcium deposits 920 and other plaque 922 is present. Calcium deposits 920 and/or other plaque 922 may be observed at various locations within a vessel and may be part of other structures. As shown in FIG. 9, a large region of other plaque 922 is shown completely lining the vessel wall 930. As described, this other plaque 922 may include fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various dead blood cells or other tissues, or other materials which may build up over time.” ¶ [0096]: “The vessel border 930 and lumen border 932 are additionally identified by the deep learning network as shown in FIG. 9.” ¶ [0096]: “The machine learning algorithm 700 may be trained to identify the vessel border and lumen border 732.” ¶ [0100]: “In some embodiments, various outputs 730 of the machine learning algorithm 700 may be determined based on other outputs 730. For example, in some embodiments, the calcium depth 733, calcium thickness 734, calcium arc 735, and calcium distribution 736 may be determined based on the identified calcium and plaque locations 731 and vessel and lumen borders 732. For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques such as [list image processing techniques here]. In other embodiments, the processor circuit may employ a deep learning algorithm to generate the outputs 733-737. The deep learning algorithm used may be the same algorithm as the machine learning algorithm 700 described with reference to FIG. 7, the convolutional neural network 800 described with reference to FIG. 8, or may be a separate deep learning algorithm.” ¶ [0101]: “As shown in FIG. 10, locations within the image 1000 corresponding to calcium deposits 1020, other plaque 1022, the vessel border 1030, and the lumen border 1032 are identified. These features may be identified according to principles similar to those described with reference to the deep learning network algorithm of FIG. 7, the convolutional neural network of FIG. 8, or methods described with reference to FIG. 9.” ¶ [0117]: “As shown in FIG. 7, additional outputs 730 of the machine learning algorithm 700 may include scan line scores 739 and a visual representation of calcium location 740. The scan line scores 739 and visual representation of calcium location 740 may be direct outputs of the machine learning algorithm 700 and independent of any other outputs 730. However, in some embodiments, the scan lines scores 739 and visual representation of calcium location 740 may be determined by the processor circuit 510 (FIG. 5) based on various outputs 730, such as the calcium and other plaque locations 731, the vessel border and lumen border 732, calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and/or calcium length 737.” ¶ [0181]: “Referring now to FIG. 17, at step 1710, the method 1700 includes receiving intravascular data from an intravascular catheter. For example, the processor circuit may receive IVUS imaging data from an IVUS imaging catheter. At step 1720, the method 1700 includes . For example, the processor circuit may identify calcium based on the IVUS imaging data.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG and RAJGURU, it would have been obvious to one of ordinary skill in the art to have modified the instructions in the computer-readable storage device taught by STEINBERG so as to also include instructions for using a machine learning model to infer the plurality of masks from the plurality of images, as taught by RAJGURU.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over STEINBERG et al. (US 2014/0094691) in view of RAJGURU et al. (US 2023/0190224), further in view of SLAGER (US 5,771,895).
Regarding claim 5 (depends on any one of claim 1 to claim 4), whereas STEINBERG and RAJGURU are not explicit as to, SLAGER teaches that aligning the plurality of images based on the plurality of masks comprises:
deriving, for each of the plurality of images (e.g., col. 12, line 37: “each image”), a frame alignment parameter (e.g., col. 12, line 38: “Z-axis” col. 12, line 53: “the Z-axis”) (col. 12, lines 27-53: “Following the acquisition, the S-VHS images are semiautomatically processed (Roelandt et al., "Three dimensional reconstruction of intracoronary ultrasound images - Rationale, approaches, problems and directions," Circ. 90(2):1044-1055, (1994)) to detect the contour of both the lumen and the external elastic membrane. Only images containing the R-peak marker are selected, thus restricting analysis to the end-diastolic shape of the artery. The X, Y positions of the detected points are transferred to a file, together with the R--R interval. The time at which each image was recorded is interpreted as Z-axis value, making use of the fact that the intravascular ultrasound catheter moved at constant speed through the artery. Using MATLAB, a numeric computation and visualization program, the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images. The contour data are subsequently converted to cylindrical coordinates, thus expressing the vessel wall, as well as the external elastic membrane by r= f(Φ, z), where r=radius; Φ =angle, and z=position along Z-axis. Next, a smoothing algorithm is applied to the surface f(Φ, z) by removing the higher frequency components of its 2D Fourier transform. The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”); and
resampling, by the computing device, the plurality of image frames (e.g., col. 12, 41-45: “the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images.”) based on the frame alignment parameters (e.g., col. 12, 41-45: “the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images.” col. 12, lines 51-53: “The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”) (col. 12, lines 27-53: “Following the acquisition, the S-VHS images are semiautomatically processed (Roelandt et al., "Three dimensional reconstruction of intracoronary ultrasound images - Rationale, approaches, problems and directions," Circ. 90(2):1044-1055, (1994)) to detect the contour of both the lumen and the external elastic membrane. Only images containing the R-peak marker are selected, thus restricting analysis to the end-diastolic shape of the artery. The X, Y positions of the detected points are transferred to a file, together with the R--R interval. The time at which each image was recorded is interpreted as Z-axis value, making use of the fact that the intravascular ultrasound catheter moved at constant speed through the artery. Using MATLAB, a numeric computation and visualization program, the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images. The contour data are subsequently converted to cylindrical coordinates, thus expressing the vessel wall, as well as the external elastic membrane by r= f(Φ, z), where r=radius; Φ =angle, and z=position along Z-axis. Next, a smoothing algorithm is applied to the surface f(Φ, z) by removing the higher frequency components of its 2D Fourier transform. The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG, RAJGURU and SLAGER, it would have been obvious to one of ordinary skill in the art to have modified the system taught by the combination of STEINBERG and RAJGURU to also include deriving a frame alignment parameter for each of the plurality of images and resampling the plurality of images based on the frame alignment parameters, as taught by SLAGER.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over STEINBERG et al. (US 2014/0094691) in view of RAJGURU et al. (US 2023/0190224), further in view of SLAGER (US 5,771,895), still further in view of KITAMURA (US 2012/0083696).
Regarding claim 18 (depends on claim 13), whereas STEINBERG and RAJGURU are not explicit as to, SLAGER teaches that the instructions when executed by the processor further cause the apparatus to:
derive, for each of the plurality of images (e.g., col. 12, line 37: “each image”), a frame alignment parameter (e.g., col. 12, line 38: “Z-axis” col. 12, line 53: “the Z-axis”) (col. 12, lines 27-53: “Following the acquisition, the S-VHS images are semiautomatically processed (Roelandt et al., "Three dimensional reconstruction of intracoronary ultrasound images - Rationale, approaches, problems and directions," Circ. 90(2):1044-1055, (1994)) to detect the contour of both the lumen and the external elastic membrane. Only images containing the R-peak marker are selected, thus restricting analysis to the end-diastolic shape of the artery. The X, Y positions of the detected points are transferred to a file, together with the R--R interval. The time at which each image was recorded is interpreted as Z-axis value, making use of the fact that the intravascular ultrasound catheter moved at constant speed through the artery. Using MATLAB, a numeric computation and visualization program, the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images. The contour data are subsequently converted to cylindrical coordinates, thus expressing the vessel wall, as well as the external elastic membrane by r= f(Φ, z), where r=radius; Φ =angle, and z=position along Z-axis. Next, a smoothing algorithm is applied to the surface f(Φ, z) by removing the higher frequency components of its 2D Fourier transform. The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”); and;
resample the plurality of image frames (e.g., col. 12, 41-45: “the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images.”) based on the frame alignment parameters (e.g., col. 12, 41-45: “the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images.” col. 12, lines 51-53: “The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”) (col. 12, lines 27-53: “Following the acquisition, the S-VHS images are semiautomatically processed (Roelandt et al., "Three dimensional reconstruction of intracoronary ultrasound images - Rationale, approaches, problems and directions," Circ. 90(2):1044-1055, (1994)) to detect the contour of both the lumen and the external elastic membrane. Only images containing the R-peak marker are selected, thus restricting analysis to the end-diastolic shape of the artery. The X, Y positions of the detected points are transferred to a file, together with the R--R interval. The time at which each image was recorded is interpreted as Z-axis value, making use of the fact that the intravascular ultrasound catheter moved at constant speed through the artery. Using MATLAB, a numeric computation and visualization program, the collected points are resampled at a constant Z-axis interval of 1 second, which translates into 1 millimeter distances between successive cross-sectional images. The contour data are subsequently converted to cylindrical coordinates, thus expressing the vessel wall, as well as the external elastic membrane by r= f(Φ, z), where r=radius; Φ =angle, and z=position along Z-axis. Next, a smoothing algorithm is applied to the surface f(Φ, z) by removing the higher frequency components of its 2D Fourier transform. The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”);
generate a vessel volume (e.g., FIG. 5; col. 12, line 56: “3-D reconstruction of the shape of the vessel,” col. 5, lines 62-64: “FIG. 5 is a 3-D view of the stack of smoothed IVUS cross-sections (Z-axis not to scale). Dark surface: external elastic membrane; light surface: vessel lumen;”) from the aligned plurality of images (e.g., col. 12, lines 51-53: “The final result is a stack of IVUS cross-sectional images (FIG. 5), all aligned perpendicular to and centered around the Z-axis.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG, RAJGURU and SLAGER, it would have been obvious to one of ordinary skill in the art to have modified the instructions in the computer-readable storage device taught by the combination of STEINBERG and RAJGURU to also include instructions for deriving a frame alignment parameter for each of the plurality of images, resampling the plurality of image frames based on the frame alignment parameters, and generating a vessel volume from the aligned plurality of images, as taught by SLAGER.
STEINBERG, RAJGURU and SLAGER fail to disclose: determine, for each voxel of the vessel volume, a color based on the key features; and render a three-dimensional (3D) visualization of the vessel volume using the determined colors.
However, KITAMURA teaches:
determine, for each voxel of the vessel volume, a color based on the key features (¶ [0116]: “In the present embodiment, as illustrated in FIG. 11B, the display control means 68 sets a different color and transparency (opacity) to voxels constituting each of a blood vessel lumen region 10a, a soft plaque region 10b and a hard plaque region 10c, which have been separately identified, with respect to the region 10A included in the correlated range of the blood vessel. Therefore, the display control means 68 can display each of the regions in an identifiable manner.” ¶ [0005]: “Further, in IntraVascular UltraSound (IVUS) diagnosis, a method, such as VH-IVUS (Virtual Histology (Registered Trademark) IntraVascular Ultrasound), has been proposed. Unlike conventional IVUS, which displays monochrome images, components are displayed in different colors in the VH-IVUS. Specifically, the tissue composition of plaque is classified into four components, namely, fibrous tissue, fibrofatty tissue, calcified tissue, and necrotic tissue by analyzing ultrasonic high frequency signals to display the components in different colors.”); and
render a three-dimensional (3D) visualization of the vessel volume (e.g., ¶ [0115]: “volume rendering image (reconstruction image) Img1” having ¶ [0115]: “the voxel value of each voxel constituting a specific structure region obtained from the 3D-IVUS image V2” … “projected into a part 10A of the coronary artery 10.”) using the determined colors ( ) (¶ [0115]: “FIG. 11A is a diagram illustrating an example of a displayed volume rendering image (reconstruction image) Img1, reconstructed from the projection three-dimensional image V3. As illustrated in FIG. 11A, the volume rendering image Img1 represents the whole heart and a coronary artery 10 reconstructed from a three-dimensional image V1 obtained by CT. Further, the voxel value of each voxel constituting a specific structure region obtained from the 3D-IVUS image V2 has been projected into a part 10A of the coronary artery 10. FIG. 11B is a diagram illustrating display of image Img2a, which is a magnified image of region Img1a in the volume rendering image Img1.” ¶ [0116]: “In the present embodiment, as illustrated in FIG. 11B, the display control means 68 sets a different color and transparency (opacity) to voxels constituting each of a blood vessel lumen region 10a, a soft plaque region 10b and a hard plaque region 10c, which have been separately identified, with respect to the region 10A included in the correlated range of the blood vessel. Therefore, the display control means 68 can display each of the regions in an identifiable manner.” ¶ [0096]: “Here, the three-dimensional tubular-structure-image, such as the 3D-IVUS image, is a three-dimensional image reconstructed by stacking tomographic images one on another. The tomographic images are obtained by imaging while a catheter having an imaging device arranged at the leading end thereof is rotated at a constant rotation speed in the tubular structure and moved at a constant speed along the longitudinal direction of the tubular structure at the same time.” ¶ [0084]: “Specifically, the value of each of voxels constituting a cross section is compared with a predetermined threshold value to judge whether they represent plaque or lumen. Further, a region composed of voxels that have been judged as plaque is identified as a plaque region, and a region composed of voxels that have been judged as lumen is identified as a lumen region. Further, with respect to the plaque, judgment is made as to whether the plaque is soft plaque or hard plaque.” ¶ [0085]: “Further, the structure extraction means 63 extracts a tubular structure also from the three-dimensional intra-tubular-structure image V2. The three-dimensional intra-tubular-structure image V2 is composed of plural two-dimensional images obtained by performing tomography, along a path in the blood vessel, in a direction orthogonal to the path. The structure extraction means 63 detects the outline of the blood vessel (the outer wall of the blood vessel) in each of original two-dimensional tomographic images. The outline is detected by using a known segmentation technique, such as Graph-Cuts, in a manner similar to the three-dimensional image V1. Further, the blood vessel region is divided into soft plaque, hard plaque and a lumen region. Then, the center of gravity of the segmented blood vessel region is set as a center position of the blood vessel. The center positions of the blood vessel in the two-dimensional tomographic images are continuously connected to each other to obtain a path in the tubular structure.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG, RAJGURU, SLAGER and KITAMURA, it would have been obvious to one of ordinary skill in the art to have modified the instructions in the computer-readable storage device taught by the combination of STEINBERG, RAJGURU and SLAGER to also include instructions for determining a color based on the key features for each voxel of the vessel volume and rendering a three-dimensional (3D) visualization of the vessel volume using the determined colors, as taught by KITAMURA.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over STEINBERG et al. (US 2014/0094691) in view of RAJGURU et al. (US 2023/0190224), further in view of LI et al. (US 2020/0226422, hereinafter referred to as “LI”).
Regarding claim 19 (depends on claim 17), whereas STEINBERG may not be entirely explicit as to, RAJGURU further teaches:
wherein the key features include at least vessel borders (¶ [0078]: “vessel borders and lumen borders 732,”), calcified plaque (¶ [0078]: “calcium plaque locations 731,” ¶ [0095]: “calcium deposits 920”), uncalcified plaque (¶ [0078]: “plaque locations 731,” ¶ [0095]: “other plaque 922” ), lumen borders (¶ [0078]: “vessel borders and lumen borders 732,”) and a stent (¶ [0037]: “stents”) (¶ [0037]: “In addition to natural structures, the device 102 may be used to examine man-made structures such as, but without limitation, heart valves, stents, shunts, filters and other devices.” ¶ [0076]: “A region 612 is also shown in FIG. 6. In the image 600 shown, the region 612 may identify a location within the vessel 620 where plaque, or another type of obstruction is present.” ¶ [0076]: “For example, an obstruction may include plaque, such as calcified deposits (e.g. calcium), dense calcium, fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various blood cells, dead blood cells, muscle cells, or other materials. An obstruction may include any or all of these materials in various compositions.” ¶ [0078]: “The machine learning algorithm 700 may be trained to identify calcium and plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, a calcium arc 735, calcium distribution 736, calcium length 737, a frame score 738, scanline scores 739, and to generate a visual representation of calcium location 740. In some embodiments, the calcium arc 735 or any other identified metric may be a secondary metric derived from inputs such as calcium plaque locations 731, vessel borders and lumen borders 732, calcium depth 733, calcium thickness 734, etc. In some embodiments, the machine learning algorithm 700 may determine that multiple deposits of calcium identified within a particular IVUS image are to be included in the same calcium arc 735.” ¶ [0095]: “As shown in FIG. 7, the deep learning network may be trained to identify calcium and plaque locations 731 within a received intravascular image 710. Similarly, as shown in FIG. 9, the system 100 may identify regions within the image 600 where calcium deposits 920 and other plaque 922 is present. Calcium deposits 920 and/or other plaque 922 may be observed at various locations within a vessel and may be part of other structures. As shown in FIG. 9, a large region of other plaque 922 is shown completely lining the vessel wall 930. As described, this other plaque 922 may include fibrous tissues, fibro-fatty tissues, lipids, complex carbohydrates, necrotic core, various dead blood cells or other tissues, or other materials which may build up over time. This build-up of other plaque 922 may restrict blood flow through the vessel. Depending on the composition of the plaque 922, the plaque 922 may exhibit different characteristics. For example, the plaque 922 may be hardened or flexible, may be more prone to movement or may be stationary, or may be prone to increasing or decreasing in size and/or severity. The presence of calcium deposits 920 within regions of other plaque 922 as shown in FIG. 9 may influence any of those characteristics and may alter the proper method of treatment.” ¶ [0096]: “The vessel border 930 and lumen border 932 are additionally identified by the deep learning network as shown in FIG. 9.” ¶ [0096]: “The machine learning algorithm 700 may be trained to identify the vessel border and lumen border 732. The processor circuit 510 may be configured to recognize the boundaries of the vessel within the ultrasound image as well as the boundaries of the lumen.” ¶ [0176]: “For example, the processor circuit may associate the deployment of a stent with an increase in the distance of a particular calcium deposit to the center of a lumen by a set amount or percentage.” ¶ [0100]: “In some embodiments, various outputs 730 of the machine learning algorithm 700 may be determined based on other outputs 730. For example, in some embodiments, the calcium depth 733, calcium thickness 734, calcium arc 735, and calcium distribution 736 may be determined based on the identified calcium and plaque locations 731 and vessel and lumen borders 732. For instance, the identified calcium and plaque locations 731 and vessel border and lumen border 732 may be overlaid over a received intravascular image. The processor circuit 510 may then be configured to use the received image with the overlay identifying calcium deposits, other plaque formations, and vessel and lumen borders to determine calcium depth 733, calcium thickness 734, calcium arc 735, calcium distribution 736, and calcium length 737. The processor circuit 510 may be configured to employ various image processing techniques such as [list image processing techniques here]. In other embodiments, the processor circuit may employ a deep learning algorithm to generate the outputs 733-737. The deep learning algorithm used may be the same algorithm as the machine learning algorithm 700 described with reference to FIG. 7, the convolutional neural network 800 described with reference to FIG. 8, or may be a separate deep learning algorithm.”) and
wherein each of the key features are associated with a different color (¶ [0040]: “Using different colors and patterns, the calcium distribution ring quickly identifies for the physician the location and extent of calcification shown in the image.” ¶ [0135]: “solid calcium, may be assigned the color white.” ¶ [0135]: “parts of the region 1252 are white corresponding to parts of the sector 1270 of solid calcium” ¶ [0147]: “parts of the region 1252 may be other colors corresponding to parts of the sector 1270 of other calcium compositions, including no calcium presence.” ¶ [0147]: “the user of the system 100 may modify or customize which colors, patterns, or other visual attributes are associated with different measurements of the image 1000.”).
STEINBERG and RAJGURU fail to explicitly disclose: a stent associated with a different color.
However, LI teaches:
wherein the key features include at least vessel borders (¶ [0028]: “characteristics and materials of the blood vessel”; ¶ [0074]: “portions of a blood vessel.” ¶ [0098]: “the lumen, intima, media and plaque are detected and identified as having boundaries corresponding to these different tissues.” ¶ [0104]: “intima I, plaque Q, adventitia ADV, imaging probe P, media M,” ¶ [0173]: “vessel wall.” ¶ [0174]: “cross-sectional and longitudinal views of a blood vessel generated using collected intravascular data.” ¶ [0174]: “the blood vessel including lumen contours, vessel diameters, vessel cross-sectional areas,” ¶ [0230]: “sidebranch SB (border is shown).” ¶ [0241]: “generate a blood vessel representation,”), calcified plaque (¶ [0013]: “atherosclerotic plaque,” ¶ [0095]: “calcium deposits 920”), uncalcified plaque (¶ [0013]: “atherosclerotic plaque,” ¶ [0095]: “other plaque 922” ), lumen borders (¶ [0013]: “lumen,” ¶ [0202]: “lumen contour”; ¶ [0204]: “lumen contour or border”) and a stent (¶ [0013]: “stent,”) (¶ [0013]: “In one embodiment, the features, regions, types, and/or classes include one or more side branch, lumen, guidewire, stent strut, stent, jailed stents, bioresorbable vascular scaffold (BVS), drug eluting stents (DES), blooming artifact, pressure wire, guidewire, lipid, atherosclerotic plaque, stenosis, calcium, calcified plaque, calcium containing tissue, lesions, fat, malapposed stent; underinflated stent; over inflated stent; radio opaque marker, branching angle of arterial tree; calibration element of probe, doped films; light scattering particles, sheath; doped sheath; fiducial registration points, diameter measure, calcium arc measure, thickness of region or feature of interest, radial measure, guide catheter, shadow region, guidewire segment, length, and thickness and others as disclosed herein.” ¶ [0101]: “arteries have various layers arranged in a consistent structure that include the intima, media and adventitia.” ¶ [0104]: “The ROI is shown as generally example and could correspond to calcium or another feature of interest such as region containing a side branch or a stent strut. Each region/feature corresponding to lumen L, intima I, plaque Q, adventitia ADV, imaging probe P, media M, and others may be generated by the MLS using a trained NN such as a CNN.” ¶ [0166]: “In addition, a mask has been generated using a trained neural network to show Calcium (Ca/red color), media (M/green color) and lumen (L/blue color). A simulated stent is also shown in a longitudinal view of an arterial representation in user interface shown in FIG. 13B.”) and
wherein each of the key features are associated with a different color (¶ [0104]: “each feature or class identified, such as ADV, EEL, IEL, L, P, I, Q may be generated as a mask or a predictive mask using one or more of the trained neural networks disclosed herein. Indicia corresponding to the output results can be show using color coded indicia and other indicia.” ¶ [0151]: “As shown, input image data, such as OCT polar image data, in example shown, is operated upon by trained neural network 34. The input polar OCT image are being operated upon to generated predictions/predictive outputs in the form of polar images that include color coded indicia corresponding to various arterial features/regions, such as Media (green), Calcium (red), and Lumen (blue) and as shown.” ¶ [0230]: “FIG. 17 shows tissue map of FIG. 16 with cross-sectional frames R1, R2, and R3 shown with media M, lumen L, and Calcium C identified. In one embodiment, these regions of interest are color coded, such as with lumen as blue, Calcium as red, and media as green. In turn, FIG. 18 shows a tissue map 1490 that includes indicia for guidewire GW (gray color), media M (green color), calcium Ca (red color), lipid LP (blue color), and sidebranch SB (border is shown). These various representations may be used to support various workflows and diagnostics.” ¶ [0228]: “As show in FIG. 16, the tissue map display 1425, pixels having a green color G indicate the area where media has been detected in tissue characterization, while red color indicate the presence of calcium plaque Ca. The colors referenced in a given tissue map user interface may vary and may be replaced by hatching in some instances or by using other indicia.” ¶ [0201]: “The masks showing predicted results are displayed in FIG. 10B.” ¶ [0202]: “image with inference or predictive results in the form of image masks showing red calcium plaque regions, blue lumen regions, and green media regions.”).
Thus, in order to obtain a more versatile and user friendly system having the cumulative features and/or functionalities taught by STEINBERG, RAJGURU and LI, it would have been obvious to one of ordinary skill in the art to have modified the apparatus taught by the combination of STEINBERG and RAJGURU to also include a key feature including a stent associated with a different color, as taught by LI.
Conclusion
At present, it is not apparent to the examiner which part of the application could serve as a basis for new and allowable claims. However, should the applicant nevertheless regard some particular matter as patentable, the examiner encourages applicant to appropriately amend the claims to include such matter and to indicate in the REMARKS the difference(s) between the prior art and the claimed invention as well as the significance thereof.
Furthermore, should applicant decide to amend the claims, examiner respectfully requests that the applicant please indicate in the REMARKS from which page(s), line(s) or claim(s) of the originally filed application that any amendments are derived. See MPEP § 2163(II)(A) (There is a strong presumption that an adequate written description of the claimed invention is present in the specification as filed, Wertheim, 541 F.2d at 262, 191 USPQ at 96; however, with respect to newly added or amended claims, applicant should show support in the original disclosure for the new or amended claims.).
A shortened statutory period for reply to this action is set to expire THREE MONTHS from the mailing date of this action. Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action. Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 USC § 133).
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINCENT PEREN who can be reached by telephone at (571) 270-7781, or via email at vincent.peren@uspto.gov. The examiner can normally be reached on Monday-Friday from 10:00 A.M. to 6:00 P.M.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KING POON, can be reached at telephone number (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/VINCENT PEREN/
Examiner, Art Unit 2617
/KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights