DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants amendments filed 06/30/2026 has been entered. Currently claims 1-17 and 19-22 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1, 4-6, 8, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen et al., (US20210038921A1) in view of Zvuloni et al., (US20130090554A1) and further in view of Rulkov et al., (US 20210128268 A1), Davies et al., (US20190008412A1) and Rao et al., (US20190148011A1)
Regarding claim 1, Nguyen teaches a system comprising (Abstract a system for guiding radiation therapy system):
at least one processor (fig. 1 computer 30 with processors [0065]); and
a memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising (fig. 1 computer 30 has memory that is capable of performing instructions [0065] and [0045]):
receiving a first data set for one or more markers ([0074] training images include markers), wherein the first data set comprises at least one of: marker image data ([0074] training images include markers).;
using the first data set to train an artificial intelligence (Al) model ([0071] a CNN is used to detect fiducial markers in kilovoltage images [0073]-[0074] training the CNN uses the training images);
receiving a second data set for a deployed marker (fig. 3 at step 304, the image is captured by the imaging system 16 [0090]);
collecting, using the imaging device, a set of images for a site of the deployed marker ([0096] there are multiple images that are received, and [0086] are of the position of the marker );
providing the second data set and the set of images for a site to the trained Al model ([0095] the image is classified by the trained CNN and the process repeated until all the images of the site are searched); and
processing the set of images as the set of images are collected using the trained Al model to identify, in real-time, the deployed biopsy in one or more images of the set of images marker based on the second data set ([0095] the image is classified as either marker or marker; [0014] the marker may be tracked in real time which means the set of images are processed in real-time).
However, Nguyen is silent regarding an ultrasound imaging device; a display, wherein images captured by the ultrasound imaging device are displayed on the display; the marker being a biopsy marker.
In the same marker field of endeavor, Zvuloni teaches an ultrasound imaging device ([0173] the imaging modality may be an ultrasound probe); a display, wherein images captured by the ultrasound imaging device are displayed on the display ([0176] display 114a displays the first images 89 a, which are the images captured by the ultrasound imaging modality ([0173]); a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers and display of Nguyen with the biopsy fiducial markers and display of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]). In addition, it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the imaging system of Nguyen with the ultrasound system of Zvuloni, as both inventions relate to the imaging of markers, and would yield the predictable result of a system that uses ultrasound to image markers and process those markers to one of ordinary skill. Indeed, Zvuloni uses different imaging modalities such as ultrasound and x-ray interchangeably (see Zvuloni [0173]). One of ordinary skill would be able to make such a substitution, and the results of Nguyen using an ultrasound imaging system are reasonably predictable.
However, the combination of references are silent regarding generating on the display in real-time with processing the one or more images, the one or more images of the set of images including an indication the deployed biopsy marker has been identified.
In the same marker imaging field of endeavor, Rulkov teaches generating on the display in real-time with processing the set of images ([0091] the surgeon may observe the display in real time), the one or more images of the set of images including an indication the deployed biopsy marker has been identified (fig. 1 the system identifies which mark is currently being localized by changing a color [0079]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the identifying marker step of modified Nguyen with the next step of including an indication of the identified markers by coloring them as taught by Rulkov, as this would help facilitate planning a procedure by helping the surgeon identify the location of the markers and therefore the lesions (see Rulkov [0083]-[0084]).
However, the combination of references are silent regarding wherein identifying the deployed biopsy marker comprises extracting, by the trained AI model, at least one acoustic characteristic of the deployed biopsy marker from the second data set and comparing the at least one acoustic characteristic against image features in the set of images to locate the deployed biopsy marker.
In the same marker imaging field of endeavor, Davies teaches wherein identifying the marker comprises extracting, by the trained AI model, at least one acoustic characteristic ([0080]markers with different acoustic properties) of the marker from the second data set ([0070] machine learning techniques are used for pattern recognition) and comparing the at least one acoustic characteristic against image features in the set of images to locate the marker ([0086]databank contains marker signatures or shapes/pattern signatures that are compared to the images from the device).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the biopsy markers of modified Nguyen to use the AI model to identify the markers as taught by Davies, as this would , as this would help recognize medical devices (see Davies [0006]).
However, the combination of references are still silent regarding in response to identifying the deployed biopsy marker, automatically transmitting, by the trained AI model, a command to the ultrasound imaging device causing the ultrasound imaging device to apply at least one image enhancement technique to the one or more images of the set of images in which the deployed biopsy marker has been identified, wherein the image enhancement technique alters at least one imaging characteristic of the one or more images in the region of the identified deployed biopsy marker.
In the same ultrasound field of endeavor, Rao teaches in response to identifying the object ([0064] the object is identified), automatically transmitting, by the trained AI model, a command to the ultrasound imaging causing the ultrasound imaging device to apply at least one image enhancement technique to the one or more images of the set of images in which the object has been identified ([0069] after identification of the object, the image is annotated, and is done without having the user add the annotation), wherein the image enhancement technique alters at least one imaging characteristic of the one or more images in the region of the identified deployed biopsy marker ([0069] the annotation can be highlighting).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the processing of the biopsy markers of modified Nguyen with the automatic image enhancement process done on objects as taught by Rao, as this would improve healthcare efficiency (see Rao [0002]). One of ordinary skill would understand that the process done on objects as taught by Rao would be applied to the biopsy markers as taught by modified Nguyen, and therefore teach the limitation “in response to identifying the deployed biopsy marker, automatically transmitting, by the trained AI model, a command to the ultrasound imaging device causing the ultrasound imaging device to apply at least one image enhancement technique to the one or more images of the set of images in which the deployed biopsy marker has been identified, wherein the image enhancement technique alters at least one imaging characteristic of the one or more images in the region of the identified deployed biopsy marker”.
Regarding claim 4, modified Nguyen teaches the system of claim 1, wherein Nguyen further teaches wherein training the Al model comprises at least one of collecting one or more 2D images for the one or more biopsy markers ([0074] the training images are 2 dimensional with dimensions of 1024x768 pixels and contain markers).
Regarding claim 5, modified Nguyen teaches the system of claim 1, wherein Nguyen further teaches wherein the deployed biopsy marker is one of the one or more biopsy markers used to train the Al model ([0080]-[0081] the CNN were trained with cylindrical gold fiducial markers and the deployed markers were also the gold fiducial markers).
Regarding claim 6, modified Nguyen teaches the system of claim 1, wherein the second data set comprises at least one of a shape ([0090] the image captured by the imaging system would have a shape of the fiducial marker).
Regarding claim 8, modified Nguyen teaches the system of claim 1, wherein Nguyen further teaches wherein the trained Al model is implemented by an imaging device configured to collect one or more images relating to the site of the deployed marker (fig. 1 imaging system 16 captures the kV image, that includes the deployed fiducial marker site [0090]).
However, Nguyen is silent regarding the marker being a biopsy marker.
In the same marker field of endeavor, Zvuloni teaches a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers of Nguyen with the biopsy fiducial markers of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]).
Regarding claim 19, Nguyen teaches a method comprising ([0071] method for training and using AI model):
receiving, by an imaging system, characteristics for a marker ([0074] AI model is trained with kV images, which include characteristics of a marker), wherein the characteristic comprises at least two of: a shape description of the biopsy marker and an image of the biopsy marker ([0080] the Cylindrical Shaped Marker Data are used to train the AI model, which includes the shape (cylindrical or arbitrary) and the [0074] kV image))
providing the received characteristics to a trained artificial intelligence (Al) component associated with the imaging system, wherein the trained Al component is trained to detect the marker when the marker is deployed in a deployment site ([0074] the kV images are used to train the AI model, and would have markers that are deployed at a deployment site (see [0080]; [0095] the image is classified by the trained CNN);
receiving, by the imaging system, one or more images of the deployment site ([0090] a kV image is captured by imaging system 16; [0093] the images include where the markers are);
processing, by the imaging system, one or more images of the deployment site as the one or more images are acquired by ( [0045] processor receives image of target area and analyses the images [0095] the CNN classifies them as marker or background):
providing the one or more images to the trained Al component ([0094]-[0095] the image is fed to the CNN, which is trained);
comparing, by the trained AI component, the one or more images to the received characteristics ([0095] the CNN classifies the normalized images. Since the CNN is trained by the training images as taught in [0074] and [0080], the CNN is indirectly comparing the training images to the images of the normalized images of [0095]);
based on comparison, identifying, by the AI component the marker in at least one of the set of images of the deployment site in real-time ([0095] the CNN classifies them as marker or background[0014] the marker may be tracked in real time).
However, Nguyen is silent regarding the marker being a biopsy marker, and using an ultrasound imaging device.
In the same marker field of endeavor, Zvuloni teaches a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]), and using an ultrasound imaging device ([0173] the imaging modality may be an ultrasound probe);.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers and display of Nguyen with the biopsy fiducial markers and display of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]). In addition, it would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to substitute the imaging system of Nguyen with the ultrasound system of Zvuloni, as both inventions relate to the imaging of markers, and would yield the predictable result of a system that uses ultrasound to image markers and process those markers to one of ordinary skill. Indeed, Zvuloni uses different imaging modalities such as ultrasound and x-ray interchangeably (see Zvuloni [0173]). One of ordinary skill would be able to make such a substitution, and the results of Nguyen using an ultrasound imaging system are reasonably predictable.
However, the combination of references are silent regarding generating on the display in real-time with processing the one or more, the one or more images of the set of images including an indication the deployed biopsy marker has been identified.
In the same marker imaging field of endeavor, Rulkov teaches generating on the display in real-time with processing the one or more images ([0091] the surgeon may observe the display with the model in real-time), the one or more images of the set of images including an indication the deployed biopsy marker has been identified (fig. 1 the system identifies which mark is currently being localized by changing a color [0079]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the identifying marker step of modified Nguyen with the next step of including an indication of the identified markers by coloring them as taught by Rulkov, as this would help facilitate planning a procedure by helping the surgeon identify the location of the markers and therefore the lesions (see Rulkov [0083]-[0084]).
However, the combination of references are silent regarding wherein identifying the deployed biopsy marker comprises extracting, by the trained AI model, at least one acoustic characteristic of the deployed biopsy marker from the second data set and comparing the at least one acoustic characteristic against image features in the set of images to locate the deployed biopsy marker.
In the same marker imaging field of endeavor, Davies teaches wherein comparing comprises applying, by the AI component([0070] machine learning techniques are used for pattern recognition), a property matching technique that uses at least one acoustic property of the marker derived from the received characterizes to match a representation of the marker against features in one or more images ([0080]markers with different acoustic properties; [0070] machine learning techniques are used for pattern recognition;[0086]databank contains marker signatures or shapes/pattern signatures that are compared to the images from the device).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the biopsy markers of modified Nguyen to use the AI model to identify the markers as taught by Davies, as this would , as this would help recognize medical devices (see Davies [0006]).
However, the combination of references are still silent regarding upon identifying the deployed biopsy marker, automatically transmitting, by the trained AI model, a command to the ultrasound imaging device causing the ultrasound imaging device to apply at least one image enhancement technique to the one or more images of the set of images in which the deployed biopsy marker has been identified, wherein the image enhancement technique alters at least one imaging characteristic of the one or more images in the region of the identified deployed biopsy marker.
In the same ultrasound field of endeavor, Rao teaches upon identifying the object ([0064] the object is identified), automatically transmitting, by the trained AI model, a control instruction to imaging hardware of the imaging system, the control instruction modifying an operational state of the imaging hardware with respect to image acquisition at the deployment site([0066] operation of the ultrasound scanner is altered after identification of the object).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the processing of the biopsy markers of modified Nguyen with the modifying of the operational state of the imaging as taught by Rao, as this would improve healthcare efficiency (see Rao [0002]). One of ordinary skill would understand that the process done on objects as taught by Rao would be applied to the biopsy markers as taught by modified Nguyen, and therefore teach the limitation “upon identifying the biopsy marker, automatically transmitting, by the AI component, a control instruction to imaging hardware of the imaging system, the control instruction modifying an operational state of the imaging hardware with respect to image acquisition at the deployment site”.
Claims 2 and 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, and Rao as applied to claim 1 and 8 above, respectively, and further in view of Henry et al., (US20220000491A1).
Regarding claim 2, modified Nguyen teaches the system of claim 1, but fails to explicitly disclose wherein the marker property information comprises at least one of shape.
In the same position marker field of endeavor, Henry teaches wherein the marker property information comprises at least one of shape ([0087] “Each marker has a predefined size, shape and pattern. The algorithm knows the shape, the size and optionally the pattern of the marker.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the first data set of modified Nguyen to include the shape of the marker property information as taught by Henry, as this improves its position and its orientation in a reference frame for each image acquisition (see Henry [0088]).
Regarding claim 9, modified Nguyen teaches the system of claim 8, wherein Nguyen further teaches wherein using the trained Al model to identify the deployed biopsy marker comprises ([0071] a CNN is used to detect fiducial markers in kilovoltage images [0073]-[0074] training the CNN uses the training images): providing the set of images to the trained Al model ([0074] the images are used to train the CNN); and evaluating, by the trained Al model in real-time, the set of images to detect identified by the second data set ([0095] the image is classified as either marker or marker; [0014] the marker may be tracked in real time).
However, Nguyen is silent regarding the marker being a biopsy marker.
In the same marker field of endeavor, Zvuloni teaches a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers of Nguyen with the biopsy fiducial markers of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]).
However the combination of references still fails explicitly disclose detecting a shape, wherein the evaluating includes the use of an image comparison algorithm.
In the same marker field of endeavor, Henry teaches detecting a shape ([0087] the image detects the shape of the marker), wherein the evaluating includes the use of an image comparison algorithm ([0087] the algorithm compares the digital representative of the stored elements, and compares it to the shape of the marker on the image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the first data set of modified Nguyen to include the shape comparison as taught by Henry, as this improves its position and its orientation in a reference frame for each image acquisition (see Henry [0088]).
Regarding claim 10, modified Nguyen teaches the system of claim 9, but fails to explicitly disclose when the image comparison algorithm detects the shape in the set of images.
However, in the same image comparison field of endeavor, Henry teaches detecting the shape in the set of images, ([0087] the image detects the shape of the marker),).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the first data set of modified Nguyen to include the shape comparison as taught by Henry, as this improves its position and its orientation in a reference frame for each image acquisition (see Henry [0088]).
However the combination of reference is silent regarding an indication of the detected marker is generated.
In the same marker imaging field of endeavor, Rulkov teaches an indication of the marker is generated (fig. 1 the system identifies which mark is currently being localized by changing a color [0079]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the detecting shapes part of the algorithm of modified Nguyen with the step of including an indication of the identified markers by coloring them as taught by Rulkov, as this would help facilitate planning a procedure by helping the surgeon identify the location of the markers and therefore the lesions (see Rulkov [0083]-[0084]).
Regarding claim 11, modified Nguyen teaches the system of claim 10, but fails to explicitly disclose wherein generating the display indication of the detected shape comprises at least one of: highlighting the detected shape in the set of images.
In the same marker field of endeavor, Rulkov teaches wherein generating the display indication of the detected shape comprises at least one of: highlighting the detected shape in the set of images (fig. 1 the system identifies which mark is currently being localized by changing a color [0079]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the detecting shapes part of the algorithm of modified Nguyen with the step of including an indication of the identified markers by coloring them as taught by Rulkov, as this would help facilitate planning a procedure by helping the surgeon identify the location of the markers and therefore the lesions (see Rulkov [0083]-[0084]).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, and Rao as applied to claim 1 above, and further in view Daon et al., (US20220265387A1).
Regarding claim 3, modified Nguyen teaches the system of claim 1, wherein Nguyen further teaches training the AI model ([0073]-[0074] training the CNN uses the training images); but fails to explicitly disclose correlating a shape of the one or more biopsy markers with corresponding marker identification information for the one or more biopsy markers.
In the same imaging a marker field of endeavor, Daon teaches a shape of the one or more biopsy markers with corresponding marker identification information for the one or more biopsy markers ([0159] each marker has a unique shape/marking which may be used to identify a marker).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the AI training of modified Nguyen to include unique markers that correlate with an identity of a marker as taught by Daon, as this would advance the effectiveness of surgical procedures (see Daon [0003]). This would allow for users to more easily distinguish between different markers. One of ordinary skill in the art would understand that Daon’s teachings of the unique marker identifiers can be applied to the training of the AI model of modified Nguyen, resulting in the reading of the limitation wherein training the Al model comprises enabling the Al model to correlate a shape of the one or more biopsy markers with corresponding marker identification information for the one or more biopsy markers.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, and Rao as applied to claim 1 above, and further in view of Chan et al., (US20150324522A1).
Regarding claim 7, modified Nguyen teaches the system of claim 1, but fails to explicitly disclose wherein the second data set is collected from at least one of: a patient record.
In the same biopsy field of endeavor, Chan teaches wherein the second data set is collected from at least one of: a patient record ([0012] medical records can contain information regarding the biopsy marker, which may also include a body part image associated with the biopsy marker).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the second data set of modified Nguyen with the patient record of Chan, as this would allow for users to more easily conceptualize diagnoses and locations referenced in medical information (see Chan [0008]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen in view of Zvuloni and in further view of Henry, Rulkov, Davies, and Rao.
Regarding claim 12, Nguyen teaches a method comprising ([0071] method for training and using AI model):
receiving, by an imaging system, a first data set for a marker ([0074] AI model is trained with kV images), wherein the first data set comprises a shape description of the marker and an identifier for the marker ([0080] the Cylindrical Shaped Marker Data and the [0082] Arbitrarily Shaped Marker Data are used to train the AI model, which includes the shape (cylindrical or arbitrary) and the identifier(cylindrical or arbitrary))
providing the first data set to an artificial intelligence (Al) component associated with the imaging system, wherein the first data is used to train the Al component to detect the marker when the marker is deployed in a deployment site ([0074] the kV images are used to train the AI model, and would have markers that are deployed at a deployment site (see [0080]);
receiving, by an imaging system, a second data set for the marker ([0090] a kV image is captured by imaging system 16);
providing the second data set to the Al component ([0094]-[0095] the image is fed to the CNN);
receiving, by the imaging system, a set of images of the deployment site ([0095] the steps are repeated until the entire image tracking window has been searched);
processing, by the imaging system, the set of images as the set of images are acquired by ([0045] processor receives images of target area and analyses them; [0095] the CNN classifies them as marker or background; [0014] the marker may be tracked in real time):
providing the second data set and the set of imaging for a site to the trained Al model ([0095] the image is classified by the trained CNN and the process repeated until all the images of the site are searched);
based on the second data set, using the Al component to identify the marker in the set of images of the deployment site in real-time ([0095] the CNN classifies them as marker or background; [0014] the marker may be tracked in real time).
However, Nguyen is silent regarding the marker being a biopsy marker.
In the same marker field of endeavor, Zvuloni teaches a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers of Nguyen with the biopsy fiducial markers of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]).
However the combination of references are silent regarding wherein the second data set comprises at least one of the shape descriptions of the biopsy marker.
However in the same marker field of endeavor, Henry teaches wherein the second data set comprises at least one of the shape descriptions of the biopsy marker (fig. 3 the fiducial markers are identified by the computer 15 by detecting the circular shape of the localizer 1 [0034]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the second data set of modified Nguyen to include the shape comparison as taught by Henry, as this improves its position and its orientation in a reference frame for each image acquisition (see Henry [0088]).
However, the combination of references are silent regarding generating on the display in real-time with processing the set of images, the one or more images of the set of images including an indication the deployed biopsy marker has been identified.
In the same marker imaging field of endeavor, Rulkov teaches generating on the display in real-time with processing the set of images ([0091] the surgeon can observe the model on the display in real time), the one or more images of the set of images including an indication the deployed biopsy marker has been identified (fig. 1 the system identifies which mark is currently being localized by changing a color [0079]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the identifying marker step of modified Nguyen with the next step of including an indication of the identified markers by coloring them as taught by Rulkov, as this would help facilitate planning a procedure by helping the surgeon identify the location of the markers and therefore the lesions (see Rulkov [0083]-[0084]).
However, the combination of references are silent regarding wherein identifying the deployed biopsy marker comprises extracting, by the trained AI model, at least one acoustic characteristic of the deployed biopsy marker from the second data set and comparing the at least one acoustic characteristic against image features in the set of images to locate the deployed biopsy marker.
In the same marker imaging field of endeavor, Davies teaches wherein identifying the marker comprises extracting, by the trained AI model, at least one acoustic characteristic ([0080]markers with different acoustic properties) of the marker from the second data set ([0070] machine learning techniques are used for pattern recognition) and comparing the at least one acoustic characteristic against image features in the set of images to locate the marker ([0086]databank contains marker signatures or shapes/pattern signatures that are compared to the images from the device).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the biopsy markers of modified Nguyen to use the AI model to identify the markers as taught by Davies, as this would , as this would help recognize medical devices (see Davies [0006]).
However, the combination of references are still silent regarding in response to identifying the biopsy marker, automatically issuing, by the AI component, a command to imaging hardware of the imaging system, the command causing the imaging hardware to alter at least one imaging parameter applied at the deployment site, thereby modifying images of the set of images to enhance detection of the identified biopsy marker.
In the same ultrasound field of endeavor, Rao teaches in response to identifying the object ([0064] the object is identified), automatically issuing, by the AI component, a command to imaging hardware of the imaging system, the command causing the imaging hardware to alter at least one imaging parameter applied at the deployment site ([0069] after identification of the object, the image is annotated, and is done without having the user add the annotation), thereby modifying images of the set of images to enhance detection of the object ([0069] the annotation can be highlighting).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the processing of the biopsy markers of modified Nguyen with the automatic image enhancement process done on objects as taught by Rao, as this would improve healthcare efficiency (see Rao [0002]). One of ordinary skill would understand that the process done on objects as taught by Rao would be applied to the biopsy markers as taught by modified Nguyen, and therefore teach the limitation “in response to identifying the biopsy marker, automatically issuing, by the AI component, a command to imaging hardware of the imaging system, the command causing the imaging hardware to alter at least one imaging parameter applied at the deployment site, thereby modifying images of the set of images to enhance detection of the identified biopsy marker”.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Henry, Rulkov, Davies, and Rao as applied to claim 12 above, and further in view of Izmirli et al., (US20190328482A1).
Regarding claim 13, modified Nguyen teaches the method claim 12, but fails to explicitly disclose generating an image of the identified biopsy marker; and displaying the image on a display device.
In the same marker imaging field of endeavor, Izmirli teaches generating an image of the identified biopsy marker; and displaying the image on a display device (fig. 2 images of the fiducial markers are displayed on display 34 [0021]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the method of modified Nguyen with the display of Izmirli, as both inventions relate to imaging markers, and would yield the predictable result of an image processing system with a display to display a generated image of markers to one of ordinary skill in the art. One of ordinary skill in the art would be able make such a combination, and the results of the method of modified Nguyen using a display to display a generated image of the marker are reasonably predictable.
However, Nguyen is silent regarding the marker being a biopsy marker.
In the same marker field of endeavor, Zvuloni teaches a biopsy marker (fig. 15C fiducial marker are visible under clinical imaging modalities, enabling recording positions of sources of each biopsy sample taken, therefore these fiducial markers are biopsy markers [0212]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the markers of Nguyen with the biopsy fiducial markers of Zvuloni, as this would provide useful information with regards to the apparent positions of objects on an image (see Zvuloni [0030]).
Claims 14 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Henry, Davies, Rao and Izmirli as applied to claim 13 above, and further in view of Riley (US20120259230A1).
Regarding claim 14, modified Nguyen teaches the method of claim 13, but fails to explicitly disclose generating the image comprises using an image enhancement technique to enhance at least a portion of the image.
In a reference reasonably pertinent to the problem of image processing faced by the inventor, Riley teaches of generating the image comprises using an image enhancement technique to enhance at least a portion of the image (fig. 4 the fiducial marker may be displayed with different colors [0063]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Nguyen with the colors of Riley, as this would provide clear references for adjusting color balances on a display (see Riley [0064]).
Regarding claim 15, modified Nguyen teaches the method of claim 14, but fails to explicitly disclose wherein the image enhancement technique comprise at least one of: modifying a color of the portion of the image.
In a reference reasonably pertinent to the problem of image processing faced by the inventor, Riley teaches wherein the image enhancement technique comprise at least one of: modifying a color of the portion of the image. (fig. 4 the fiducial marker may be displayed with different colors [0063]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Nguyen with the colors of Riley, as this would provide clear references for adjusting color balances on a display (see Riley [0064]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Henry, Rulkov, Davies, and Rao and Izmirli as applied to claim 13 above, and further in view of Chan.
Regarding claim 16, modified Nguyen teaches the method of claim 13, but fails to explicitly disclose wherein generating the image comprises adding information associated with the marker to the image, the information comprising at least one of: marker attributes or a marker detection confidence score.
However in the same marker imaging field of endeavor, Kawagishi teaches wherein generating the image comprises adding information associated with the marker to the image, the information comprising at least one of: marker attributes ([0054] the mapping tool may overlay a symbol on a body part image, which the symbol is based on diagnosis data associated with the marker, thus giving the marker attributes).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the second data set of modified Nguyen with the symbol overlay of Chan, as this would allow for user to more easily conceptualize diagnoses and locations referenced in medical information (see Chan [0008]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Henry, Rulkov, Davies, and Rao as applied to claim 14 above, and further in view of Yamada et al., (US20190111282A1).
Regarding claim 17, modified Nguyen teaches the method of claim 14, wherein Nguyen further teaches using the AI component to identify the biopsy marker in the set of images ([0095] the CNN classifies them as marker or background), but fails to explicitly disclose using one or more image matching techniques to match an image representation of the biopsy marker to data in the set of images.
In the same marker imaging field of endeavor, Yamada teaches using one or more image matching techniques to match an image representation of the biopsy marker to data in the set of images (fig. 3 the template image of the marker 29 is compared with the captured image [0043]; [0039] multiple images are taken).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the AI component of modified Nguyen to use the image matching technique of Yamada, as this would reduce the frequency of erroneous detection of an object to be tracked (see Yamada [0013]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Henry, Rulkov, Davies, Rao, and Yamada as applied to claim 17 above, and further in view of Kumagai
Regarding claim 18, modified Nguyen teaches the system of claim 17, but fails to explicitly disclose when a match between the image representation of the biopsy marker and the data in the set of images is detected, an indication of the match is provided by the imaging system.
However, in the same image comparison field of endeavor, Kumagai teaches when a match between the image representation of the biopsy marker and the data in the set of images is detected, an indication of the match is provided by the imaging system (fig. 4 the image comparison unit 55 compares the two images to calculate a match [0068]; the visual check result reception unit 56 receives an indication of a match or not match, which may be based on a matching degree value, such as 1.0 [0090]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to combine the image comparison algorithm of modified Nguyen with the image comparison unit of Kumagai, as both inventions relate to image processing, and would yield the predictable result of an image comparison algorithm comparing shapes, and an indication of the detected shape is generated to one of ordinary skill in the art. One of ordinary skill in the art would be able to perform such as combination, and the results of modified Nguyen incorporating the image comparison unit of Kumagai are reasonably predictable.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, and Rao as applied to claim 19 above, and further in view of Seip et al., (US20160317129A1).
Regarding claim 20, modified Nguyen teaches the method of claim 19, but fails to explicitly disclose wherein the one or more images of the deployment site are exported to an alternative imaging system.
In the same image processing field of endeavor, Seip teaches wherein the one or more images of the deployment site are exported to an alternative imaging system ([0050] the images of the fiducial markers may be transferred to [0051] an ultrasound image system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Nguyen to include another imaging system as taught by Seip, as this would help to provide a predictive treatment outcome value of STL treatments (see Seip [0068]).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, Rao, and Seip as applied to claim 20 above, and further in view of Dyer et al., (US20170265947A1).
Regarding claim 21, modified Nguyen teaches the method of claim 20, but fails to explicitly disclose wherein the alternative imaging system is a multimodal device configured to perform real-time detection of the marker.
In the same marker field of endeavor, Dyer teaches the alternative imaging system is a multimodal device configured to perform real-time detection of the marker (fig. 23 the system 216 is obtains a real-time multi-modal image that can track a fiducial marker 212 [0180]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Nguyen with the multi-modal imaging system of Dyer, as this would help facilitate navigation of a medical procedure (see Dyer Abstract).
Claims 22 is rejected under 35 U.S.C. 103 as being unpatentable over Nguyen as modified by Zvuloni, Rulkov, Davies, and Rao as applied to claim 1, and further in view of Ellis et al., (US20170105709A1).
Regarding claim 22, modified Nguyen teaches the system of claim 1, but is silent regarding wherein the acoustic characteristic comprises at least one of: an echogenicity of the deployed biopsy marker relative to surrounding tissue.
In the same biopsy marker field of endeavor, Ellis teaches wherein the acoustic characteristic comprises at least one of: an echogenicity of the deployed biopsy marker relative to surrounding tissue ([0015] the different fiducial markers can be distinguished by image intensity, which on an ultrasound image would be relative echogenicity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Nguyen with the acoustic characteristic of Ellis, as this would improve clinical outcomes (see Ellis [0002]).
Response to Arguments
Applicant's arguments filed 06/30/2026 regarding the 35 U.S.C. 103 rejection have been fully considered but they are not persuasive.
Regarding claim 1, Applicant argues that Nguyen is directed to using kV x-ray imaging and does not include acoustic properties, and is a physical mechanism entirely unlike the reflection of acoustic waves in ultrasound. Applicant further argues the combination of references do not teach or disclose the amended claim 1. Examiner disagrees with this argument, as one of ordinary skill in the art would be able perform the same type of image processing of Nguyen done on x-ray on ultrasound. Further, Nguyen in view of Zvuloni and Rulkov are not used to teach the amended limitations, and instead it is the combination of Nguyen in view of Zvuloni, Rulkov, Davies, and Rao has been used to teach claim 1.
The remaining claims are rejected for substantially the same reason as above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y FANG whose telephone number is (571)272-0952. The examiner can normally be reached Mon - Friday 9:30 am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 5712722714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL YIMING FANG/Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
