DETAILED ACTION
This office action is in response to the communication received on January 30, 2026
concerning application No. 18/836,900 filed on August 8, 2024.
Claims 1-2, 4-11, 15, 17-18, 27, 43, and 48-52 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on January 30, 2026 has been entered.
Response to Arguments
In response to applicant’s remarks regarding the interview summary. Examiner notes that as outlined in the Examiner Interview Summary dated 01/15/2026 only option 1 provided by applicant was agreed upon as appearing to overcome the prior art of record. Option 2, in the examiner’s opinion is taught by the Bechtold reference relied upon in the previous office action.
Applicant's arguments filed 01/30/2026 regarding the 35 USC 112 rejection have been fully considered. The amendments to the claims have been entered and overcome the 35 USC 112 rejection of claim 49, previously set forth.
Applicant's arguments filed 01/30/2026 regarding the 35 USC 103 rejections have been fully considered but they are not persuasive. In response to the applicant’s arguments that the prior art fails to teach “wherein each of the plurality of training images is a respective ultrasound image a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions of a brain, wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object under a gyral fold of the brain”, examiner respectfully disagrees. As set forth in the previous office action, Gong teaches each of the plurality of training images is a respective ultrasound image of a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions, wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object in the respective one of the plurality of regions. [0036] and [0149] of Gong further teaches the region of interest being imaged includes the brain. Bechtold is relied upon for teaching the deficiencies of Gong. Specifically, pgs. 3-4, section 2.2 and fig. 5 of Bechtold disclose obtaining images of the cotton ball (foreign body object) being detected under a brain which includes a gyral fold of the subject. Therefore, the combination of Gong in view of Bechtold teaches “wherein each of the plurality of training images is a respective ultrasound image a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions of a brain, wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object under a gyral fold of the brain”. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the known technique of having the foreign body object be under a gyral fold of the brain of Bechtold to the training images of Gong to allow for the predictable results of improving the quality and ability of the convolutional neural network in detecting a foreign body object within the image.
Claim Objections
Claims 1, 4, 6-7, 18, 27, and 50 are objected to because of the following informalities:
Claim 1, line 10, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 4, line 2, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 6, line 2, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 6, line 3, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 7, line 2, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 18, lines 12-13, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 27, lines 26-27, “the plurality of regions” should read “the plurality of regions of the brain”,
Claim 50, line 2, “the plurality of regions” should read “the plurality of regions of the brain”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-6, 18, 27, and 48-50 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gong et al. (US 20200211186, hereinafter Gong) in view of Bechtold et al. (“Minimizing cotton retention in neurosurgical procedures: which imaging modality can help?, hereinafter Bechtold) and Anderson et al. (US 20220020142, hereinafter Anderson).
Regarding claim 1, Gong teaches a method of forming a trained model in which one or more processing devices perform operations(Abstract and [0031] disclose training a detection model) comprising:
receiving a plurality of training images, each training image in the plurality of training images associated with a respective training boundary data set of a plurality of training boundary data sets ([0031] “ the detection model may be generated by training a preliminary detection model based on a plurality of training images. The preliminary detection model may be a deep learning model. The deep learning model may have multiple layers to learn training detection results of the training images using a set of algorithms”. [0033] further discloses obtaining a plurality of training images. [0161] discloses the training images include detections such as a mask or ground truth corresponding to the location of the training object (boundary data) within the image), each training image in the plurality of training images further represented as a respective training image data set of a plurality of training image data sets ([0159] discloses generating a detection result using each training image, the combination of the detection result and the training image for each image is considered the respective training image data set);
wherein each of the plurality of training images is a respective ultrasound image of a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions of a brain ([0028] discloses the imaging modality is an ultrasound system and [0152] discloses the training image is an ultrasound image. [0159] further discloses generating a detection result for each of the images which corresponds to associating the image with a respective region. [0036] and [0149] disclose the region of interest of the subject is the brain), wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object in the respective one of the plurality of regions ([0055] discloses the exemplary object detected in the images is a foreign body);
processing the plurality of training image data sets using a convolutional neural network to generate a plurality of output boundary data sets and to select a plurality of training weights ([0156] discloses obtaining a preliminary detection model which is a convolutional neural network. [0159] further discloses obtaining training detection result for each of the images by inputting the training images into the preliminary detection model. [0132] discloses allocating weights to the different detection results);
wherein the convolutional neural network comprises (i) a plurality of layers with a plurality of weights ([0157]-[0158] discloses the preliminary detection model includes plurality of layers and a set of weights within the range of -1 to 1); and
wherein, when using the convolutional neural network to generate the plurality of output boundary data sets the plurality of training weights are selected to minimize a loss function between the plurality of output boundary data sets and the plurality of respective training boundary data sets ([0117] “the range may be determined based on a plurality of sample images (e.g., 493 sample images). The second detection result of an image may include a box of a shape enclosing one or both lungs of a subject. The box may be of the shape of a polygon, a circle, an ellipse, etc., defined by a boundary or contour. In some embodiments, the range may be defined by one or more distance thresholds between the boundary or contour of the box and one or more edges of the image (or a region within the image)”. [0165]-[0166], “the comparison result may assess a difference between the training detection result(s) and the reference detection result(s). In some embodiments, the processing device 140B may determine an objective function based on the difference as the comparison result. For example, the objective function may include a loss function of the difference, a Root Mean Square Error (RMSE) function, a Mean Absolute Error (MAE) function, etc. As illustrated in connection with FIG. 5, the processing device 140B may generate the detection model by performing a plurality of iterations to iteratively update the one or more training parameters of the preliminary detection model”); and fixing the plurality of training weights to form the trained model when the loss function is minimized ([0089] “An exemplary termination condition may relate to an objective function (e.g., a loss function, a total variation function) relating to the plurality of preliminary or intermediate detection sub-models”).
Gong does not specifically teach the respective foreign body object comprises a cotton ball and that the foreign body object is under a gyral fold of the brain.
However,
Bechtold in a similar field of endeavor teaches a respective foreign body object comprises a cotton ball (pg. 1, Abstract and pgs. 3-4, Ultrasound Imaging, disclose obtaining ultrasound images of a cotton ball within an object being imaged) and the foreign body is under a gyral fold of the brain (pgs. 3-4, section 2.2 and fig. 5 disclose the cotton ball (foreign body) is detected under a brain which includes a gyral fold).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the foreign object of Gong for the cotton ball of Bechtold because it amounts to simple substitution of one known element for another to obtain the predictable results of identifying whether foreign objects such as cotton balls are present within the object being imaged.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the known technique of having the foreign body object be under a gyral fold of the brain of Bechtold to the training images of Gong to allow for the predictable results of improving the quality and ability of the convolutional neural network in detecting a foreign body object within the image.
Gong in view of Bechtold does not specifically teach the convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights.
However,
Anderson in a similar field of endeavor teaches a convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights ([0109] “pre-training is used for part of a model. For example, pre-training may be used for certain layers of the model”. [0112] further discloses training comprises updating the weights, meaning the neural network includes a plurality of pre-trained weights), and (ii) a plurality of training and appended layers with the plurality of training weights ([0109] discloses pre-training is performed to inform updates (append) to the model parameters (layers and weights)); wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed ([0114] “an output of stage 50 in version of the first neural network 72 having updated weights, the weights of the first neural network 72 are fixed at the end of stage 50”); and wherein fixing the plurality of training weights further selects a plurality of fixed training weights ([0145]-[0146] “The alternation of stages 56 and 58 is repeated until convergence is reached. When convergence is reached, stage 54 is terminated. The weights for the first neural network 72 and second neural network 82 are fixed. At stage 60, the training circuitry 44 outputs a trained first neural network having the weights that were fixed at the end of stage 54, and a trained second neural network having the weights that were fixed at the end of stage 54”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the convolutional neural network disclosed by Gong in view of Bechtold to comprise (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights in order to improve the performance of the neural network, as recognized by Anderson ([0161]).
Regarding claim 2, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Gong further teaches the loss function is a mean squared error function using a plurality of error values, each error value of the plurality of error values being equal to a respective numerical difference between at least one of the plurality of output boundary data sets and a respective one of the plurality of training boundary data sets ([0165] “the comparison result may assess a difference between the training detection result(s) and the reference detection result(s). In some embodiments, the processing device 140B may determine an objective function based on the difference as the comparison result. For example, the objective function may include a loss function of the difference, a Root Mean Square Error (RMSE) function, a Mean Absolute Error (MAE) function, etc.”).
Regarding claim 4, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Gong further teaches the at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective one of the plurality of training boundary data sets, such that the respective one of the plurality of training boundary data sets is a set of number values associated with a bounding box enclosing the respective foreign body object ([0161] “a reference detection result of a training object may include a position of the training object, a bounding box of the training object”. The coordinates of the bounding box are considered the set number of values. Also see [0176]).
Regarding claim 5, Gong in view of Bechtold and Anderson teaches the method of claim 4, as set forth above. Gong further teaches the bounding box enclosing the respective foreign body object is a ground truth bounding box ([0161] “the reference detection result(s) may also be referred to as one or more masks or ground truths corresponding to one or more training objects in the first training image”).
Regarding claim 6, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Gong further teaches at least one of the plurality of ultrasound images is associated with a respective one of the plurality of regions such that the respective one of the plurality of regions contains no foreign body object ([0098] “the imaging standard for an image of object(s) includes one or more of: the image lacking any foreign object”).
Regarding claim 18, Gong teaches a system for forming a trained model (system 100 in fig. 1. [0073] discloses training a detection model), the system comprising:
a non-transitory computer readable storage medium associated with a computing device, the non-transitory computer readable storage medium storing program instructions executable by the computing device to cause the computing device to perform operations ([0186] “A non-transitory computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave… A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device”) comprising:
receiving a plurality of training images, each training image in the plurality of training images associated with a respective training boundary data set of a plurality of training boundary data sets ([0031] “ the detection model may be generated by training a preliminary detection model based on a plurality of training images. The preliminary detection model may be a deep learning model. The deep learning model may have multiple layers to learn training detection results of the training images using a set of algorithms”. [0033] further discloses obtaining a plurality of training images. [0161] discloses the training images include detections such as a mask or ground truth corresponding to the location of the training object (boundary data) within the image), each training image in the plurality of training images further represented as a respective training image data set of a plurality of training image data sets ([0159] discloses generating a detection result using each training image, the combination of the detection result and the training image for each image is considered the respective training image data set);
wherein each of the plurality of training images is a respective ultrasound image of a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions of a brain ([0028] discloses the imaging modality is an ultrasound system and [0152] discloses the training image is an ultrasound image. [0159] further discloses generating a detection result for each of the images which corresponds to associating the image with a respective region. [0036] and [0149] discloses the region of interest includes the brain), wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object in the respective one of the plurality of regions ([0055] discloses the exemplary object detected in the images is a foreign body);
processing the plurality of training image data sets using a convolutional neural network to generate a plurality of output boundary data sets and to select a plurality of training weights ([0156] discloses obtaining a preliminary detection model which is a convolutional neural network. [0159] further discloses obtaining training detection result for each of the images by inputting the training images into the preliminary detection model. [0132] discloses allocating weights to the different detection results);
wherein the convolutional neural network comprises (i) a plurality of layers with a plurality of weights ([0157]-[0158] discloses the preliminary detection model includes plurality of layers and a set of weights within the range of -1 to 1); and
wherein, when using the convolutional neural network to generate the plurality of output boundary data sets the plurality of training weights are selected to minimize a loss function between the plurality of output boundary data sets and the plurality of respective training boundary data sets ([0117] “the range may be determined based on a plurality of sample images (e.g., 493 sample images). The second detection result of an image may include a box of a shape enclosing one or both lungs of a subject. The box may be of the shape of a polygon, a circle, an ellipse, etc., defined by a boundary or contour. In some embodiments, the range may be defined by one or more distance thresholds between the boundary or contour of the box and one or more edges of the image (or a region within the image)”. [0165]-[0166], “the comparison result may assess a difference between the training detection result(s) and the reference detection result(s). In some embodiments, the processing device 140B may determine an objective function based on the difference as the comparison result. For example, the objective function may include a loss function of the difference, a Root Mean Square Error (RMSE) function, a Mean Absolute Error (MAE) function, etc. As illustrated in connection with FIG. 5, the processing device 140B may generate the detection model by performing a plurality of iterations to iteratively update the one or more training parameters of the preliminary detection model”); and fixing the plurality of training weights to form the trained model when the loss function is minimized ([0089] “An exemplary termination condition may relate to an objective function (e.g., a loss function, a total variation function) relating to the plurality of preliminary or intermediate detection sub-models”.
Gong does not specifically teach the respective foreign body object comprises a cotton ball and that the foreign body object is under a gyral fold of the brain.
However,
Bechtold in a similar field of endeavor teaches a respective foreign body object comprises a cotton ball (pg. 1, Abstract and pgs. 3-4, Ultrasound Imaging, disclose obtaining ultrasound images of a cotton ball within an object being imaged) and the foreign body is under a gyral fold of the brain (pgs. 3-4, section 2.2 and fig. 5 disclose the cotton ball (foreign body) is detected under a brain which includes a gyral fold).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the foreign object of Gong for the cotton ball of Bechtold because it amounts to simple substitution of one known element for another to obtain the predictable results of identifying whether foreign objects such as cotton balls are present within the object being imaged.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the known technique of having the foreign body object be under a gyral fold of the brain of Bechtold to the training images of Gong to allow for the predictable results of improving the quality and ability of the convolutional neural network in detecting a foreign body object within the image.
Gong in view of Bechtold does not specifically teach the convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights.
However,
Anderson in a similar field of endeavor teaches a convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights ([0109] “pre-training is used for part of a model. For example, pre-training may be used for certain layers of the model”. [0112] further discloses training comprises updating the weights, meaning the neural network includes a plurality of pre-trained weights), and (ii) a plurality of training and appended layers with the plurality of training weights ([0109] discloses pre-training is performed to inform updates (append) to the model parameters (layers and weights)); wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed ([0114] “an output of stage 50 in version of the first neural network 72 having updated weights, the weights of the first neural network 72 are fixed at the end of stage 50”); and wherein fixing the plurality of training weights further selects a plurality of fixed training weights ([0145]-[0146] “The alternation of stages 56 and 58 is repeated until convergence is reached. When convergence is reached, stage 54 is terminated. The weights for the first neural network 72 and second neural network 82 are fixed. At stage 60, the training circuitry 44 outputs a trained first neural network having the weights that were fixed at the end of stage 54, and a trained second neural network having the weights that were fixed at the end of stage 54”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the convolutional neural network disclosed by Gong in view of Bechtold to comprise (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights in order to improve the performance of the neural network, as recognized by Anderson ([0161]).
Regarding claim 27, Gong teaches a system (system 100 in fig. 1) comprising:
at least one processor (the electronic circuitry of system 100 in fig. 1); and
at least one non-transitory computer readable media associated with the at least one processor storing program instructions that when executed by the at least one processor cause the at least one processor to perform operations for generating a boundary data set from an input image ([0186] “A non-transitory computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave… A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device”), the operations comprising:
receiving a plurality of training images represented as an input image data set ([0031] “ the detection model may be generated by training a preliminary detection model based on a plurality of training images. The preliminary detection model may be a deep learning model. The deep learning model may have multiple layers to learn training detection results of the training images using a set of algorithms”. [0033] further discloses obtaining a plurality of training images);
processing the input image data set using a convolutional neural network to generate the boundary data set ([0156] discloses obtaining a preliminary detection model which is a convolutional neural network. [0159] further discloses obtaining training detection result for each of the images by inputting the training images into the preliminary detection model);
wherein the convolutional neural network comprises (i) a plurality of layers with a plurality of weights ([0157]-[0158] discloses the preliminary detection model includes plurality of layers and a set of weights within the range of -1 to 1);
wherein the plurality of training weights are selected according to training operations performed by one or more processors associated with one or more non-transitory computer readable media, the one or more non-transitory computer readable media storing training program instructions that when executed by the one or more processors cause the one or more processors to perform the training operations ([0186]-[0187] discloses the non-transitory computer readable medium stores instructions for carrying out operations of the present method) comprising:
receiving a plurality of training images, each training image in the plurality of training images associated with a respective training boundary data set of a plurality of training boundary data sets ([0031] “ the detection model may be generated by training a preliminary detection model based on a plurality of training images. The preliminary detection model may be a deep learning model. The deep learning model may have multiple layers to learn training detection results of the training images using a set of algorithms”. [0033] further discloses obtaining a plurality of training images. [0161] discloses the training images include detections such as a mask or ground truth corresponding to the location of the training object (boundary data) within the image), each training image in the plurality of training images further represented as a respective training image data set of a plurality of training image data sets ([0159] discloses generating a detection result using each training image, the combination of the detection result and the training image for each image is considered the respective training image data set);
wherein each of the plurality of training images is a respective ultrasound image of a plurality of ultrasound images, the respective ultrasound image further associated with a respective one of a plurality of regions of a brain ([0028] discloses the imaging modality is an ultrasound system and [0152] discloses the training image is an ultrasound image. [0159] further discloses generating a detection result for each of the images which corresponds to associating the image with a respective region. [0036] and [0149] disclose the region of interest includes a brain), wherein at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions is further associated with a respective foreign body object in the respective one of the plurality of regions ([0055] discloses the exemplary object detected in the images is a foreign body);
processing the plurality of training image data sets using a training convolutional neural network to generate a plurality of output boundary data sets and to select a plurality of training weights ([0156] discloses obtaining a preliminary detection model which is a convolutional neural network. [0159] further discloses obtaining training detection result for each of the images by inputting the training images into the preliminary detection model. [0132] discloses allocating weights to the different detection results);
wherein the training convolutional neural network comprises (i) the plurality of layers with the plurality of weights ([0157]-[0158] discloses the preliminary detection model includes plurality of layers and a set of weights within the range of -1 to 1); and
wherein, when using the training convolutional neural network to generate the plurality of output boundary data sets the plurality of training weights are selected to minimize a loss function between the plurality of output boundary data sets and the plurality of respective training boundary data sets ([0117] “the range may be determined based on a plurality of sample images (e.g., 493 sample images). The second detection result of an image may include a box of a shape enclosing one or both lungs of a subject. The box may be of the shape of a polygon, a circle, an ellipse, etc., defined by a boundary or contour. In some embodiments, the range may be defined by one or more distance thresholds between the boundary or contour of the box and one or more edges of the image (or a region within the image)”. [0165]-[0166], “the comparison result may assess a difference between the training detection result(s) and the reference detection result(s). In some embodiments, the processing device 140B may determine an objective function based on the difference as the comparison result. For example, the objective function may include a loss function of the difference, a Root Mean Square Error (RMSE) function, a Mean Absolute Error (MAE) function, etc. As illustrated in connection with FIG. 5, the processing device 140B may generate the detection model by performing a plurality of iterations to iteratively update the one or more training parameters of the preliminary detection model”); and fixing the plurality of training weights to form the trained model when the loss function is minimized ([0089] “An exemplary termination condition may relate to an objective function (e.g., a loss function, a total variation function) relating to the plurality of preliminary or intermediate detection sub-models”.
Gong does not specifically teach the respective foreign body object comprises a cotton ball and that the foreign body object is under a gyral fold of the brain.
However,
Bechtold in a similar field of endeavor teaches a respective foreign body object comprises a cotton ball (pg. 1, Abstract and pgs. 3-4, Ultrasound Imaging, disclose obtaining ultrasound images of a cotton ball within an object being imaged) and the foreign body is under a gyral fold of the brain (pgs. 3-4, section 2.2 and fig. 5 disclose the cotton ball (foreign body) is detected under a brain which includes a gyral fold).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the foreign object of Gong for the cotton ball of Bechtold because it amounts to simple substitution of one known element for another to obtain the predictable results of identifying whether foreign objects such as cotton balls are present within the object being imaged.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the known technique of having the foreign body object be under a gyral fold of the brain of Bechtold to the training images of Gong to allow for the predictable results of improving the quality and ability of the convolutional neural network in detecting a foreign body object within the image.
Gong in view of Bechtold does not specifically teach the convolutional neural network and training convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the training convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights.
However,
Anderson in a similar field of endeavor teaches a convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights ([0109] “pre-training is used for part of a model. For example, pre-training may be used for certain layers of the model”. [0112] further discloses training comprises updating the weights, meaning the neural network includes a plurality of pre-trained weights), and (ii) a plurality of training and appended layers with the plurality of training weights ([0109] discloses pre-training is performed to inform updates (append) to the model parameters (layers and weights)); wherein when using the convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed ([0114] “an output of stage 50 in version of the first neural network 72 having updated weights, the weights of the first neural network 72 are fixed at the end of stage 50”); and wherein fixing the plurality of training weights further selects a plurality of fixed training weights ([0145]-[0146] “The alternation of stages 56 and 58 is repeated until convergence is reached. When convergence is reached, stage 54 is terminated. The weights for the first neural network 72 and second neural network 82 are fixed. At stage 60, the training circuitry 44 outputs a trained first neural network having the weights that were fixed at the end of stage 54, and a trained second neural network having the weights that were fixed at the end of stage 54”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the convolutional neural network and training convolutional neural network disclosed by Gong in view of Bechtold to comprise (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights; wherein when using the training convolutional neural network to generate the plurality of output boundary data sets, the plurality of pre-trained weights are fixed; and wherein fixing the plurality of training weights further selects a plurality of fixed training weights in order to improve the performance of the neural network, as recognized by Anderson ([0161]).
Regarding claim 48, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Bechtold further teaches the cotton ball comprises a blood-soaked cotton ball (pg. 1, Abstract discloses “blood-soaked cotton balls”).
Regarding claim 49, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Gong further teaches the plurality of ultrasound images are captured in an absence of injections, dyes ([0152] disclose obtaining images using techniques other than those that use contrast agents to enhance the images, therefore the images are captured in the absences of injections/dyes), or radiofrequency tags (Examiner notes that nowhere in Gong discloses the use of radiofrequency tags being imaged) in the cranium ([0036] discloses the imaging subject includes the brain).
Bechtold further teaches the cranium is an open cranial cavity (pg. 1, Abstract, “we explored the ability to distinguish surgical cotton on or below the tissue surface from brain parenchyma using ultrasound imaging”. Fig. 1 further shows an intraoperative photo where the region is an open cranial cavity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the brain region of Gong for the open cranial cavity of Bechtold because it amounts to simple substitution of one known element for another to obtain the predictable results of being able to identify foreign objects during surgery.
Regarding claim 50, Gong in view of Bechtold and Anderson teaches the method of claim 4, as set forth above. Gong further teaches the respective one of the plurality of regions is a cranium ([0036] discloses the imaging subject includes the brain).
Bechtold further teaches the cranium is an open cranial cavity (pg. 1, Abstract, “we explored the ability to distinguish surgical cotton on or below the tissue surface from brain parenchyma using ultrasound imaging”. Fig. 1 further shows an intraoperative photo where the region is an open cranial cavity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the brain region of Gong for the open cranial cavity of Bechtold because it amounts to simple substitution of one known element for another to obtain the predictable results of being able to identify foreign objects during surgery.
Claim(s) 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gong in view of Bechtold and Anderson as applied to claim 6 above, and further in view of Piekniewski et al. (US 20180293742, hereinafter Piekniewski).
Regarding claim 7, Gong in view of Bechtold and Anderson teaches the method of claim 6, as set forth above. Gong in view of Bechtold and Anderson does not specifically teach the at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions containing no foreign body object is further associated with a respective one of the plurality of training boundary data sets, such that the respective one of the plurality of training boundary data sets is a set of number values associated with a null bounding box.
However,
Piekniewski in a similar field of endeavor teaches the at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions containing no foreign body object is further associated with a respective one of the plurality of training boundary data sets, such that the respective one of the plurality of training boundary data sets is a set of number values associated with a null bounding box ([0173] “If the determination indicates that the object may be present, the tracking process may communicate information related to the bounding box of the object in visual space coordinates. In some implementations, target object location (e.g., 402 in FIG. 4) referenced to the image may be combined with image sensor parameters (e.g., orientation in 3D, position) in order to determine bounding box location in visual space coordinates. If the object is absent, an object not found indication (e.g., a bounding box with negative coordinates and/or a null object) may be provided”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method disclosed by Gong in view of Bechtold and Anderson to have the at least one of the plurality of ultrasound images associated with the respective one of the plurality of regions containing no foreign body object is further associated with a respective one of the plurality of training boundary data sets, such that the respective one of the plurality of training boundary data sets is a set of number values associated with a null bounding box in order to provide an indication to a user that the object is not present within the image, thereby increasing the efficiency of the procedure by reducing the need of the user to determine whether the object is present or not.
Regarding claim 8, Gong in view of Bechtold, Anderson and Piekniewski teaches the method of claim 7, as set forth above. Piekniewski further teaches the set of number values associated with the null bounding box comprises: an x-coordinate value equal to zero of the null bounding box; a y-coordinate value equal to zero of the null bounding box; a width value equal to zero of the null bounding box; and a height value equal to zero of the null bounding box ([0173] “If the determination indicates that the object may be present, the tracking process may communicate information related to the bounding box of the object in visual space coordinates. In some implementations, target object location (e.g., 402 in FIG. 4) referenced to the image may be combined with image sensor parameters (e.g., orientation in 3D, position) in order to determine bounding box location in visual space coordinates. If the object is absent, an object not found indication (e.g., a bounding box with negative coordinates and/or a null object) may be provided”. Since null represents zero, the x-coordinate value, y-coordinate value, width value, and height value of the bounding box are all considered zero).
Claim(s) 9-11, 15, 17, and 51 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gong in view of Bechtold and Anderson as applied to claims 1 and 4 above, and further in view of Tuzoff et al. (US 20200146646, hereinafter Tuzoff).
Regarding claim 9, Gong in view of Bechtold and Anderson teaches the method of claim 4, as set forth above. Gong in view of Bechtold and Anderson does not specifically teach the set of number values associated with the bounding box enclosing the respective foreign body object comprises: an x-coordinate value of an upper left corner of the bounding box; a y-coordinate value of the upper left corner of the bounding box; a width value of the bounding box; and a height value of the bounding box.
However,
Tuzoff in a similar field of endeavor teaches the set of number values associated with the bounding box enclosing the respective foreign body object comprises:
an x-coordinate value of an upper left corner of the bounding box ([0031] “The system produces as output (6) the coordinates of the bounding boxes for the teeth detected in the source dental image, and corresponding teeth numbers for all detected teeth in the image” and [0057] “The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected”. The coordinate defining the rectangular region within the image include an x-coordinate value for the upper left corner of the bounding box);
a y-coordinate value of the upper left corner of the bounding box ([0031], [0057] The coordinate defining the rectangular region within the image include an y-coordinate value for the upper left corner of the bounding box);
a width value of the bounding box ([0061] discloses determining the dimensions of the bounding box which includes a width value); and
a height value of the bounding box ([0061] discloses determining the dimensions of the bounding box which includes a height value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method disclosed by Gong in view of Bechtold and Anderson to have the set of number values associated with the bounding box enclosing the respective foreign body object comprises: an x-coordinate value of an upper left corner of the bounding box; a y-coordinate value of the upper left corner of the bounding box; a width value of the bounding box; and a height value of the bounding box in order to specifically know the size and location of the bounding, therefore allowing the user to know the size and location of the region of interest within the image, thereby increasing the efficiency since the user does not have to manually determine the size and location of the region of interest.
Regarding claim 10, Gong in view of Bechtold, Anderson and Tuzoff teaches a method of generating a boundary data set from an input image in which one or more processing devices perform operations comprising: forming the trained model of claim 9 (See the rejections of claim 1 and 9 above). Gong further teaches receiving the input image represented as an input image data set ([0031] “ the detection model may be generated by training a preliminary detection model based on a plurality of training images. The preliminary detection model may be a deep learning model. The deep learning model may have multiple layers to learn training detection results of the training images using a set of algorithms”. [0033] further discloses obtaining a plurality of training images);
processing the input image data set using a convolutional neural network to generate the boundary data set ([0156] discloses obtaining a preliminary detection model which is a convolutional neural network. [0159] further discloses obtaining training detection result for each of the images by inputting the training images into the preliminary detection model. [0132] discloses allocating weights to the different detection results);
wherein the convolutional neural network comprises (i) a plurality of layers with a plurality of weights ([0157]-[0158] discloses the preliminary detection model includes plurality of layers and a set of weights within the range of -1 to 1).
Gong does not specifically teach the convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights.
However,
Anderson in a similar field of endeavor teaches a convolutional neural network comprises (i) a plurality of pre-trained layers with a plurality of pre-trained weights ([0109] “pre-training is used for part of a model. For example, pre-training may be used for certain layers of the model”. [0112] further discloses training comprises updating the weights, meaning the neural network includes a plurality of pre-trained weights), and (ii) a plurality of training and appended layers with the plurality of training weights ([0109] discloses pre-training is performed to inform updates (append) to the model parameters (layers and weights)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the convolutional neural network disclosed by Gong in view of Bechtold, Anderson, and Tuzoff to comprise (i) a plurality of pre-trained layers with a plurality of pre-trained weights, and (ii) a plurality of training and appended layers with the plurality of training weights in order to improve the performance of the neural network, as recognized by Anderson ([0161]).
Regarding claim 11, Gong in view of Bechtold, Anderson, and Tuzoff teaches the method of claim 10, as set forth above. Tuzoff further teaches the boundary data set is a set of output number values associated with an output bounding box enclosing a localized region in the input image ([0057] “The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected”. The coordinate defining the rectangular region within the image include are considered the set of output number values associated with the output bounding box), the set of output number values associated with the output bounding box enclosing the localized region comprising: an output x-coordinate value of an upper left corner of the output bounding box ([0031], [0057] The coordinates defining the rectangular region within the image include an x-coordinate value for the upper left corner of the bounding box); an output y-coordinate value of the upper left corner of the output bounding box ([0031], [0057] The coordinates defining the rectangular region within the image include an y-coordinate value for the upper left corner of the bounding box); an output width value of the output bounding box([0061] discloses determining the dimensions of the bounding box which includes a width value); and an output height value of the output bounding box ([0061] discloses determining the dimensions of the bounding box which includes a width value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method disclosed by Gong in view of Bechtold, Anderson and Tuzoff to have the boundary data set be a set of output number values associated with an output bounding box enclosing a localized region in the input image, the set of output number values associated with the output bounding box enclosing the localized region comprising: an output x-coordinate value of an upper left corner of the output bounding box; an output y-coordinate value of the upper left corner of the output bounding box; an output width value of the output bounding box; and an output height value of the output bounding box in order to specifically know the size and location of the bounding, therefore allowing the user to know the size and location of the region of interest within the image, thereby increasing the efficiency since the user does not have to manually determine the size and location of the region of interest.
Regarding claim 15, Gong in view of Bechtold, Anderson and Tuzoff teaches the method of claim 11, as set forth above. Gong further teaches the plurality of appended layers comprise at least one of: a dropout layer and a dense layer ([0085] discloses the convolutional neural network is a DenseNet model, therefore the appended layers of the model comprise at least a dense layer).
Regarding claim 17, Gong in view of Bechtold, Anderson and Tuzoff teaches the method of claim 11, as set forth above. Gong further teaches generating a representation for display on a display device, the representation for display including an overlay of a representation of the output bounding box on a representation of the input image ([0176] “A region 1220 may represent a third reference detection result correspond to a foreign object of an image 1210 as shown in FIG. 12b . The third reference detection result relates to a position of the foreign object, a bounding box of the foreign object”. See fig. 12b where a bounding box 1220 is being displayed as an overlay on the input image 1210).
Regarding claim 51, Gong in view of Bechtold and Anderson teaches the method of claim 1, as set forth above. Gong in view of Bechtold and Anderson does not specifically teach the training image data sets have a depth of 3 representing red, green, and blue values assigned to pixels, wherein the output boundary data sets are 1x4, and wherein the 4 values in the 1x4 output boundary data sets represent x and y coordinates of a bounding box enclosing the respective foreign body object as well as a width and height of the bounding box.
However,
Tuzoff in a similar field of endeavor teaches the training image data sets have a depth of 3 representing red, green, and blue values assigned to pixels ([0061] discloses marking the conditions within the image using color coding which corresponds to the image having a depth of 3 representing red, green, and blue because color coding is performed using an red, green, blue (RGB) scale), wherein the output boundary data sets are 1x4, and wherein the 4 values in the 1x4 output boundary data sets represent x and y coordinates of a bounding box enclosing the respective foreign body object as well as a width and height of the bounding box ([0031] “The system produces as output (6) the coordinates of the bounding boxes for the teeth detected in the source dental image, and corresponding teeth numbers for all detected teeth in the image” and [0057] “The updated charting data may also comprise an indication of the bounding box or region of interest for the input image as identified during either the tooth detection or condition detection by the analysis service 1000. This region of interest information may comprise coordinates defining each portion of the original image (e.g., coordinates identifying absolute pixel positions within the image, or coordinates and offsets defining a rectangular region within the image) for which a tooth and/or a condition was detected”. The coordinate defining the rectangular region within the image include an x-coordinate value and a y-coordinate value for the bounding box. [0061] further discloses determining the dimensions of the bounding box which includes a width value and a height value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the known technique of having the training image data sets have a depth of 3 representing red, green, and blue values assigned to pixels, wherein the output boundary data sets are 1x4, and wherein the 4 values in the 1x4 output boundary data sets represent x and y coordinates of a bounding box enclosing the respective foreign body object as well as a width and height of the bounding box of Tuzoff to the method disclosed by Gong in view of Bechtold and Anderson to allow for the predictable results of increasing the accuracy of the convolutional neural network, by providing more precise training data.
Claim(s) 43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gong in view of Bechtold and Anderson as applied to claim 27 above, and further in view of Krupat et al. (US 20180303397, hereinafter Krupat).
Regarding claim 43, Gong in view of Bechtold and Anderson teaches the method of claim 27, as set forth above. Gong further teaches a smartphone comprises the at least one processor, the at least one non-transitory computer readable media, and the display device ([0038] discloses the terminal includes a smart mobile device such as a smartphone).
Gong in view of Bechtold and Anderson does not specifically teach the smartphone is further configured to capture the input image, or wherein the system further comprises: a networked computer device that comprises the at least one processor and the at least one non-transitory computer readable media; and a remote computing device that comprises the display device and that is configured to transmit the input image to the networked computing device, wherein the remote computing device is further configured to capture the input image, wherein the remote computing device is a smartphone.
However,
Krupat in a similar field of endeavor teaches the smartphone is configured to capture the input image ([0097] discloses a smartphone camera is used for obtaining the image data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the smartphone disclosed by Gong in view of Bechtold and Anderson for the smartphone of Krupat because it amounts to simple substitution of one known element for another to obtain the predictable results of obtaining the input image.
Allowable Subject Matter
Claim 52 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: The prior art of record fails to reasonably teach or in combination render obvious the following limitations when the claims taken as a whole to include, “the convolutional neural network comprises, in order: a Conv2D x2, MaxPooling 2D function; a Conv2D x3, MaxPooling 2D function; a flatten function; a dense x2 function; and a dropout function, wherein convolutional layers in the convolutional neural network have a 3x3 kernal size, max pooling layers use a 2x2 stride size, and dropout is 50%”, wherein the convolutional neural network is used to generate a plurality of output boundary data sets and to select a plurality of training weights.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW BEGEMAN whose telephone number is (571)272-4744. The examiner can normally be reached Monday-Thursday 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at 5712701790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW W BEGEMAN/Examiner, Art Unit 3798