DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number CN202311688885.4 dated 12/08/2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 12/06/2024 has been considered and placed in the application file.
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1, line 6, should be “acquiring a 2D contour of the examination subject based on
Claims 2-11 depend either directly or indirectly from the objection of claim 1, therefore they are also objected.
Appropriate correction is required.
1st Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 7, 11, 12, 18, and 19 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.) in view of US Patent Publication 2021 0201476 A1, (Prasad et al.).
Claim 1
Regarding claim 1, Zheng et al. teach a method for predicting a collision between an examination subject and an imaging apparatus, comprising: acquiring an image package of the examination subject via a multi-modal camera system, ("The sensing devices 110 may be configured to generate the images," par. 17) the image package including a depth image and a thermal image of the examination subject, and the multi-modal camera system including a depth camera module and a thermal camera module; ("Each of the sensing devices 110 may include one or more sensors such as one or more 2D visual sensors (e.g., 2D cameras), one or more 3D visual sensors (e.g., 3D cameras), one or more red, green and blue (RGB) sensors, one or more depth sensors, one or more RGB plus depth (RGB-D) sensors, one or more thermal sensors … that may be configured to capture images of a person," par. 17) and acquiring a 2D contour of the examination subject ("The first model 202 may include a parametric model of the patient, a two-dimensional (2D) or three-dimensional (3D) contour of the patient," par. 32) based on of segmentation processing performed on the thermal image ("The first model 202 may be generated by the organ geometry estimator 200 or a different device or apparatus based on images (e.g., RGB images, depth images, thermal images, etc.)," par. 32).
Zheng et al. do not explicitly teach all of generating a 3D contour of the examination subject based on the 2D contour of the examination subject and the depth image of the examination subject; and estimating, based on the 3D contour of the examination subject, whether the examination subject will collide, on a movement path thereof, with an imaging apparatus scanning the examination subject.
[AltContent: textbox (Figure 5 shows the generated 3D point cloud of 3D structure based on the initial depth image and 2D image.)]However, Prasad et al. teach generating a 3D contour of the examination subject based on the 2D contour of the examination subject and the depth image of the
PNG
media_image1.png
420
564
media_image1.png
Greyscale
examination subject; ("a tonal depth image 804 of the patient covered in the blanket (e.g., in dim lighting conditions) is generated from the 2D image 802 and then used to generate a raw 3D point cloud," par. 99) and estimating, based on the 3D contour of the examination subject, whether the examination subject will collide, ("the post-processed point cloud may be used to perform scan outcome prediction," par. 88) on a movement path thereof, with an imaging apparatus scanning the examination subject ("The scan outcome prediction may include determining if any potential patient to gantry contact may occur when the patient and scan table are moved into the bore as well as predicting potential contact once scanning commences," par. 107).
Therefore, taking the teachings of Zheng et al. and Prasad et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al. to use the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al. The suggestion/motivation for doing so would have been that, “the body contour or structure of the subject 112 may be estimated using an image reconstructed from point cloud data generated by the camera image data processor 215 from images received from the depth camera 114. These subject parameters may be used by the computing device 216, for example, to perform patient-scanner contact prediction” as noted by the Prasad et al. disclosure in paragraph [0043], which also motivates combination because the combination would predictably have a higher accuracy as there is a reasonable expectation that combining 3D contour generation and contact prediction with existing 2D methods yields higher accuracy and better safety through improved patient-scanner positioning and collision avoidance; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of apparatus claim 12 while noting that the rejection above cites to both device and method disclosures. Claim 12 is mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 7
Regarding claim 7, Zheng et al. and Prasad et al. teach the method according to claim 1 as noted above.
Zheng et al. do not explicitly teach all of wherein generating a 3D contour of the examination includes: calculating 3D coordinate values of each point on the examination subject based on depth information in the depth image and pixel distance information in the 2D contour, and acquiring the 3D contour based on all the 3D coordinate values.
However, Prasad et al. teach wherein generating a 3D contour of the examination includes: calculating 3D coordinate values of each point on the examination subject based on depth information in the depth image and pixel distance information in the 2D contour, ("At 532, a raw 3D point cloud may be generated using the images captured from the depth camera. The raw 3D point cloud refers to a collection of data points defined by the 3D world coordinate system," par. 80) and acquiring the 3D contour ("isosurface volumetric extraction may be performed on the raw 3D point cloud generated at 532 to detect the shape/orientation/pose of the patient," par. 81) based on all the 3D coordinate values ("The filtered point cloud array in the gantry coordinates generated by the point cloud algorithm may be output at 416 and may then be used for 3D patient structure estimation," par. 58).
Zheng et al. and Prasad et al. are combined as per claim 1.
Claim 11
Regarding claim 11, Zheng et al. and Prasad et al. teach the method according to claim 1 as noted above.
Zheng et al. do not explicitly teach all of wherein estimating, based on the 3D contour of the examination subject includes: calculating 3D contour coordinate values of the 3D contour of the examination subject in a machine frame coordinate system of the imaging apparatus, the 3D contour coordinate values including 3D contour coordinate values of the examination subject moving to each position during scanning; and when the 3D contour coordinate values overlap with coordinate values of a machine frame hole of the imaging apparatus, determining that the examination subject will collide, on the movement path thereof, with the machine frame hole.
However, Prasad et al. teach wherein estimating, based on the 3D contour of the examination subject includes: calculating 3D contour coordinate values of the 3D contour of the examination subject in a machine frame coordinate system of the imaging apparatus, the 3D contour coordinate values including 3D contour coordinate values of the examination subject moving to each position during scanning; ("isosurface volumetric extraction may be performed on the raw 3D point cloud generated at 532 to detect the shape/orientation/pose of the patient," par. 81) and when the 3D contour coordinate values overlap with coordinate values of a machine frame hole of the imaging apparatus, determining that the examination subject will collide, on the movement path thereof, with the machine frame hole ("if the determined patient pose does match the desired patient pose, method 1000 proceeds to 1010 and includes performing a scan outcome prediction based on the patient shape relative to table coordinates. The scan outcome prediction may include determining if any potential patient to gantry contact may occur when the patient and scan table are moved into the bore as well as predicting potential contact once scanning commences," par. 107).
Zheng et al. and Prasad et al. are combined as per claim 1.
Claim 12
Regarding claim 12, Zheng et al. teach an imaging apparatus, comprising: a multi-modal camera system, including a depth camera module and a thermal camera module, ("The sensing devices 110 may be configured to generate the images," par. 17) the multi-modal camera system being configured to acquire an image package of the examination subject, the image package including a depth image and a thermal image of the examination subject, and the multi-modal camera system including a depth camera module and a thermal camera module; ("Each of the sensing devices 110 may include one or more sensors such as one or more 2D visual sensors (e.g., 2D cameras), one or more 3D visual sensors (e.g., 3D cameras), one or more red, green and blue (RGB) sensors, one or more depth sensors, one or more RGB plus depth (RGB-D) sensors, one or more thermal sensors … that may be configured to capture images of a person," par. 17) and a processing unit, ("the apparatus 600 may include a processor," par. 45) configured to: acquire a 2D contour of the examination subject ("The first model 202 may include a parametric model of the patient, a two-dimensional (2D) or three-dimensional (3D) contour of the patient," par. 32) based on segmentation processing performed on the thermal image ("The first model 202 may be generated by the organ geometry estimator 200 or a different device or apparatus based on images (e.g., RGB images, depth images, thermal images, etc.)," par. 32).
Zheng et al. do not explicitly teach all of a machine frame, including a machine frame hole for accommodating an examination subject; generate a 3D contour of the examination subject based on the 2D contour of the examination subject and the depth image of the examination subject; and estimate, based on the 3D contour of the examination subject, whether the examination subject will collide, on a movement path thereof, with an imaging apparatus scanning the examination subject.
[AltContent: textbox (Figure 1 shows the machine frame and hole.)]However, Prasad et al. teach a machine frame, including a machine frame hole
PNG
media_image2.png
350
402
media_image2.png
Greyscale
for accommodating an examination subject; generate a 3D contour of the examination subject based on the 2D contour of the examination subject and the depth image of the examination subject; ("a tonal depth image 804 of the patient covered in the blanket (e.g., in dim lighting conditions) is generated from the 2D image 802 and then used to generate a raw 3D point cloud," par. 99) and estimate, based on the 3D contour of the examination subject, whether the examination subject will collide, ("the post-processed point cloud may be used to perform scan outcome prediction," par. 88) on a movement path thereof, with an imaging apparatus scanning the examination subject ("The scan outcome prediction may include determining if any potential patient to gantry contact may occur when the patient and scan table are moved into the bore as well as predicting potential contact once scanning commences," par. 107).
Zheng et al. and Prasad et al. are combined as per claim 1.
Claim 18
Regarding claim 18, Zheng et al. and Prasad et al. teach the imaging apparatus according to claim 12 as noted above.
Zheng et al. do not explicitly teach all of wherein the processing unit is further configured to: calculate 3D coordinate values of each point on the examination subject based on depth information in the depth image and pixel distance information in the 2D contour, and acquire the 3D contour based on all the 3D coordinate values.
However, Prasad et al. teach wherein the processing unit is further configured to: calculate 3D coordinate values of each point on the examination subject ("At 532, a raw 3D point cloud may be generated using the images captured from the depth camera. The raw 3D point cloud refers to a collection of data points defined by the 3D world coordinate system," par. 80) based on depth information in the depth image and pixel distance information in the 2D contour, ("isosurface volumetric extraction may be performed on the raw 3D point cloud generated at 532 to detect the shape/orientation/pose of the patient," par. 81) and acquire the 3D contour based on all the 3D coordinate values ("The filtered point cloud array in the gantry coordinates generated by the point cloud algorithm may be output at 416 and may then be used for 3D patient structure estimation," par. 58).
Zheng et al. and Prasad et al. are combined as per claim 1.
Claim 19
Regarding claim 19, Zheng et al. and Prasad et al. teach the imaging apparatus according to claim 12 as noted above.
Zheng et al. do not explicitly teach all of wherein the processing unit is further configured to: calculate 3D contour coordinate values of the 3D contour of the examination subject in a machine frame coordinate system of the imaging apparatus, the 3D contour coordinate values including 3D contour coordinate values of the examination subject moving to each position during scanning; and when the 3D contour coordinate values overlap with coordinate values of a machine frame hole of the imaging apparatus, determine that the examination subject will collide, on the movement path thereof, with the machine frame hole.
However, Prasad et al. teach wherein the processing unit is further configured to: calculate 3D contour coordinate values of the 3D contour of the examination subject in a machine frame coordinate system of the imaging apparatus, the 3D contour coordinate values including 3D contour coordinate values of the examination subject moving to each position during scanning; ("isosurface volumetric extraction may be performed on the raw 3D point cloud generated at 532 to detect the shape/orientation/pose of the patient," par. 81) and when the 3D contour coordinate values overlap with coordinate values of a machine frame hole of the imaging apparatus, determine that the examination subject will collide, on the movement path thereof, with the machine frame hole ("if the determined patient pose does match the desired patient pose, method 1000 proceeds to 1010 and includes performing a scan outcome prediction based on the patient shape relative to table coordinates. The scan outcome prediction may include determining if any potential patient to gantry contact may occur when the patient and scan table are moved into the bore as well as predicting potential contact once scanning commences," par. 107).
Zheng et al. and Prasad et al. are combined as per claim 1.
2nd Claim Rejections - 35 USC § 103
Claims 2, 3, 13, and 14 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.) and US Patent Publication 2021 0201476 A1, (Prasad et al.) in view of US Patent Publication 2018 0046878 A1, (Wang et al.).
Claim 2
Regarding claim 2, Zheng et al. and Prasad et al. teach the method according to claim 1 as noted above.
Zheng et al. teach wherein acquiring a 2D contour of the examination subject includes: performing the segmentation processing on the thermal image ("The processing device 112 may be further configured to determine, based on all or the first subset of images, a patient model that may indicate a pose and/or body shape of the patient," par. 25) based on a plurality of predetermined temperature thresholds to acquire a plurality of thermal contour images; ("The first model 202 may be generated by the organ geometry estimator 200 or a different device or apparatus based on images (e.g., RGB images, depth images, thermal images, etc.) of the patient," par. 32) and extracting the 2D contour of the examination subject from a thermal contour image ("based on all or a second subset of images captured by the sensing device(s) 110, an environment model that may indicate a 3D spatial layout of the medical environment 100 (e.g., in terms of respective [X, Y, Z] coordinates or locations of the people and objects detected by the processing device based on the images, and/or the respective contours of the people and objects)," par. 26).
Zheng et al. do not explicitly teach all of a thermal contour image most conforming to a contour of the examination subject among the plurality of thermal contour images.
However, Wang et al. teach a thermal contour image most conforming to a contour of the examination subject among the plurality of thermal contour images ("Comparing the different segmentation coordinates with the segmentation coordinates of the greyscale image of the green component, and if the different segmentation coordinates match the segmentation coordinates of the greyscale image of the green component, retaining the different segmentation coordinates, otherwise removing them," par. 63).
Therefore, taking the teachings of Zheng et al., Prasad et al., and Wang et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al. and the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al. to use thermal image contour processing and extraction as taught by Wang et al. The suggestion/motivation for doing so would have been that, “the segmentation thresholds of the greyscale images of the red component and the blue component, thereby the efficiency of the segmentation threshold calculation can be improved, and the efficiency of the infrared thermal image contour extraction can be improved” as noted by the Wang et al. disclosure in paragraph [0051], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that the threshold calculation and contour extraction would operate with faster processing times and improved accuracy by utilizing the separated color components; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 3
Regarding claim 3, Zheng et al. and Prasad et al. teach the method according to claim 1 as noted above.
Zheng et al. teach wherein the image package of the examination subject is acquired via the multi-modal camera system in real time, wherein acquiring a 2D contour of the examination subject includes: extracting the 2D contour of the examination subject from the thermal contour image ("based on all or a second subset of images captured by the sensing device(s) 110, an environment model that may indicate a 3D spatial layout of the medical environment 100 (e.g., in terms of respective [X, Y, Z] coordinates or locations of the people and objects detected by the processing device based on the images, and/or the respective contours of the people and objects)," par. 26).
Zheng et al. do not explicitly teach all of performing segmentation processing on a current thermal image based on a preselected temperature threshold to acquire a thermal contour image.
However, Wang et al. teach performing segmentation processing on a current thermal image based on a preselected temperature threshold to acquire a thermal contour image ("the temperature of the human body is higher than the temperature of the environment, through histogram statistics of infrared thermal image, the thresholds of the edge of human body can be calculated, and the thresholds is further segmented to extract the contour of the human body, thereby the precise contour of the human body can be obtained," par. 38).
Zheng et al., Prasad et al., and Wang et al. are combined as per claim 2.
Claim 13
Regarding claim 13, Zheng et al. and Prasad et al. teach the imaging apparatus according to claim 12 as noted above.
Zheng et al. teach wherein the processing unit is further configured to: perform the segmentation processing on the thermal image ("The processing device 112 may be further configured to determine, based on all or the first subset of images, a patient model that may indicate a pose and/or body shape of the patient," par. 25) based on a plurality of predetermined temperature thresholds to acquire a plurality of thermal contour images; ("The first model 202 may be generated by the organ geometry estimator 200 or a different device or apparatus based on images (e.g., RGB images, depth images, thermal images, etc.) of the patient," par. 32) and extract the 2D contour of the examination subject from a thermal contour image ("based on all or a second subset of images captured by the sensing device(s) 110, an environment model that may indicate a 3D spatial layout of the medical environment 100 (e.g., in terms of respective [X, Y, Z] coordinates or locations of the people and objects detected by the processing device based on the images, and/or the respective contours of the people and objects)," par. 26).
Zheng et al. do not explicitly teach all of a thermal contour image most conforming to a contour of the examination subject among the plurality of thermal contour images.
However, Wang et al. teach a thermal contour image most conforming to a contour of the examination subject among the plurality of thermal contour images ("Comparing the different segmentation coordinates with the segmentation coordinates of the greyscale image of the green component, and if the different segmentation coordinates match the segmentation coordinates of the greyscale image of the green component, retaining the different segmentation coordinates, otherwise removing them," par. 63).
Zheng et al., Prasad et al., and Wang et al. are combined as per claim 2.
Claim 14
Regarding claim 14, Zheng et al. and Prasad et al. teach the imaging apparatus according to claim 12 as noted above.
Zheng et al. teach wherein the multi-modal camera system acquires the image package of the examination subject in real time, wherein the processing unit is further configured to: extract the 2D contour of the examination subject from the thermal contour image ("based on all or a second subset of images captured by the sensing device(s) 110, an environment model that may indicate a 3D spatial layout of the medical environment 100 (e.g., in terms of respective [X, Y, Z] coordinates or locations of the people and objects detected by the processing device based on the images, and/or the respective contours of the people and objects)," par. 26).
Zheng et al. do not explicitly teach all of perform segmentation processing on a current thermal image based on a preselected temperature threshold to acquire a thermal contour image.
However, Wang et al. teach perform segmentation processing on a current thermal image based on a preselected temperature threshold to acquire a thermal contour image ("the temperature of the human body is higher than the temperature of the environment, through histogram statistics of infrared thermal image, the thresholds of the edge of human body can be calculated, and the thresholds is further segmented to extract the contour of the human body, thereby the precise contour of the human body can be obtained," par. 38).
Zheng et al., Prasad et al., and Wang et al. are combined as per claim 2.
3rd Claim Rejections - 35 USC § 103
Claims 4 and 15 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.), US Patent Publication 2021 0201476 A1, (Prasad et al.), and US Patent Publication 2018 0046878 A1, (Wang et al.) in view of US Patent Publication 2021 0295517 A1, (Parrish et al.).
Claim 4
Regarding claim 4, Zheng et al., Prasad et al., and Wang et al. teach the method according to claim 3 as noted above.
Zheng et al. do not explicitly teach all of the method according to claim 3, wherein the preselected temperature threshold is acquired via the following steps: performing, based on a plurality of predetermined temperature thresholds, segmentation processing on a thermal image acquired at a certain previous time to acquire a plurality of thermal contour images; and selecting, from the plurality of thermal contour images, a thermal contour image most conforming to a contour of the examination subject, and determining a temperature threshold corresponding thereto to be the preselected temperature threshold.
However, Wang et al. teach wherein the preselected temperature threshold is acquired via the following steps: performing, based on a plurality of predetermined temperature thresholds, segmentation processing on a thermal image ("the temperature of the human body is higher than the temperature of the environment, through histogram statistics of infrared thermal image, the thresholds of the edge of human body can be calculated, and the thresholds is further segmented to extract the contour of the human body, thereby the precise contour of the human body can be obtained," par. 38) acquired at a certain previous time to acquire a plurality of thermal contour images; ("capturing the static infrared thermal video," par. 17) ("only 5-10 seconds are needed to capture 250-500 frames of image, thereby a sufficient number of images can be provided," par. 45) and selecting, from the plurality of thermal contour images, a thermal contour image most conforming to a contour of the examination subject ("Comparing the different segmentation coordinates with the segmentation coordinates of the greyscale image of the green component, and if the different segmentation coordinates match the segmentation coordinates of the greyscale image of the green component, retaining the different segmentation coordinates, otherwise removing them," par. 63).
Additionally, Parrish et al. teach determining a temperature threshold corresponding thereto to be the preselected temperature threshold ("determining that the actual temperature is above a threshold relative to a target temperature, the target temperature in the simplest case being a nominal value corresponding to expected or known normal body temperature," par. 68).
Therefore, taking the teachings of Zheng et al., Prasad et al., Wang et al., and Parrish et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al., the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al., and thermal image contour processing and extraction as taught by Wang et al. to use temperature threshold selection for thermal contour determination as taught by Parrish et al. The suggestion/motivation for doing so would have been that, “Then the temperature discrimination range can be lowered to tighter thresholds around the expected body temperature range, possibly incrementally or in one or two range decreases, until a region is identified with a sufficient number of pixels within the smallest range that still leaves at least one region meeting the size and temperature criteria” as noted by the Parrish et al. disclosure in paragraph [0074], which also motivates combination because the combination would predictably have a higher productivity as there is a reasonable expectation that the combination would yield a more accurate subject outline with less computational load; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 15
Regarding claim 15, Zheng et al., Prasad et al., and Wang et al. teach the imaging apparatus according to claim 14 as noted above.
Zheng et al. do not explicitly teach all of wherein the processing unit is further configured to: perform, based on a plurality of predetermined temperature thresholds, segmentation processing on a thermal image acquired at a certain previous time to acquire a plurality of thermal contour images; and select, from the plurality of thermal contour images, a thermal contour image most conforming to a contour of the examination subject, and determine a temperature threshold corresponding thereto to be the preselected temperature threshold.
However, Wang et al. teach wherein the processing unit is further configured to: perform, based on a plurality of predetermined temperature thresholds, segmentation processing on a thermal image ("the temperature of the human body is higher than the temperature of the environment, through histogram statistics of infrared thermal image, the thresholds of the edge of human body can be calculated, and the thresholds is further segmented to extract the contour of the human body, thereby the precise contour of the human body can be obtained," par. 38) acquired at a certain previous time to acquire a plurality of thermal contour images; ("capturing the static infrared thermal video," par. 17) ("only 5-10 seconds are needed to capture 250-500 frames of image, thereby a sufficient number of images can be provided," par. 45) and select, from the plurality of thermal contour images, a thermal contour image most conforming to a contour of the examination subject ("Comparing the different segmentation coordinates with the segmentation coordinates of the greyscale image of the green component, and if the different segmentation coordinates match the segmentation coordinates of the greyscale image of the green component, retaining the different segmentation coordinates, otherwise removing them," par. 63).
Additionally, Parrish et al. teach determine a temperature threshold corresponding thereto to be the preselected temperature threshold ("determining that the actual temperature is above a threshold relative to a target temperature, the target temperature in the simplest case being a nominal value corresponding to expected or known normal body temperature," par. 68).
Zheng et al., Prasad et al., Wang et al., and Parrish et al. are combined as per claim 4.
4th Claim Rejections - 35 USC § 103
Claim 8 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.) and US Patent Publication 2021 0201476 A1, (Prasad et al.) in view of US Patent Publication 2026 0076630 A1, (Bystrov et al.).
Claim 8
Regarding claim 2, Zheng et al. and Prasad et al. teach the method according to claim 7 as noted above.
Zheng et al. do not explicitly teach all of wherein the depth information includes a perpendicular depth from each point on the examination subject to a focal point of the depth camera module or the thermal camera module, and the pixel distance information includes a pixel distance from each pixel in the 2D contour to the focal point of the depth camera module or the thermal camera module, wherein the pixels in the 2D contour correspond to the points on the examination subject.
However, Bystrov et al. teach wherein the depth information includes a perpendicular depth from each point on the examination subject to a focal point of the depth camera module or the thermal camera module, and the pixel distance information includes a pixel distance from each pixel in the 2D contour to the focal point of the depth camera module or the thermal camera module, wherein the pixels in the 2D contour correspond to the points on the examination subject ("A depth image may for example be obtained by a range camera, which may produce a 2D image that shows the distance to points in a scene from a predetermined reference point or plane (e.g. a focal point or other reference point). Depth information may be obtained by stereo imaging, in which images are (e.g. substantially simultaneously) acquired by two (conventional, e.g. monochrome or RGB) cameras from different vantage points. Processing techniques known in the art may be applied to such stereo images (to the pair of concomitantly acquired images) to derive depth information," par. 101).
Therefore, taking the teachings of Zheng et al., Prasad et al., and Bystrov et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al. and the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al. to use the 3D surface mapping and coordinate mapping as taught by Bystrov et al. The suggestion/motivation for doing so would have been that, “From the shape and displacement of the imaged (line) reflection, the distance between the reflections (points on said line-e.g. every point in the image after fully scanning the scene) and a reference, e.g. the light source and/or camera can be relatively easily computed” as noted by the Bystrov et al. disclosure in paragraph [0103], which also motivates combination because the combination would predictably have a greater ease of use as there is a reasonable expectation that the system will provide a simpler and more efficient way to compute distances and map coordinates in 3D space by leveraging structured light or optical reflection data; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
4th Claim Rejections - 35 USC § 103
Claim 9 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.) and US Patent Publication 2021 0201476 A1, (Prasad et al.) in view of US Patent Publication 2022 0021856 A1, (Nakamura et al.).
Claim 9
Regarding claim 9, Zheng et al. and Prasad et al. teach the method according to claim 7 as noted above.
Zheng et al. do not explicitly teach all of wherein generating a 3D contour of the examination includes the thermal image or the 2D contour being converted to be in a depth camera coordinate system, or the depth image being converted to be in a thermal camera coordinate system.
However, Nakamura et al. teach wherein generating a 3D contour of the examination includes the thermal image or the 2D contour being converted to be in a depth camera coordinate system, or the depth image being converted to be in a thermal camera coordinate system ("the projection control device 50 can accurately convert the optional coordinates (Xc, Yc) in the infrared captured image IRG into the corresponding coordinates (Xp, Yp) in the projection image," par. 72).
Therefore, taking the teachings of Zheng et al., Prasad et al., and Nakamura et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al. and the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al. to use the thermal image coordinate conversion methods as taught by Nakamura et al. The suggestion/motivation for doing so would have been that, “a projection conversion matrix) of a conversion processing that determines a correspondence relationship between a position (specifically, coordinates) in the infrared captured image generated by the IR camera 40 and a position (specifically, coordinates)” as noted by the Nakamura et al. disclosure in paragraph [0069], which also motivates combination because the combination would predictably have a higher accuracy as there is a reasonable expectation that the combination would function to accurately map 2D thermal image coordinates to 3D contour positions, ensuring precise alignment between thermal data and 3D surface models for better contact prediction; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
5th Claim Rejections - 35 USC § 103
Claim 10 is rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2024 0099774 A1, (Zheng et al.), US Patent Publication 2021 0201476 A1, (Prasad et al.), and US Patent Publication 2022 0021856 A1, (Nakamura et al.) in view of US Patent Publication 2021 0295517 A1, (Parrish et al.).
Claim 10
Regarding claim 10, Zheng et al., Prasad et al., and Nakamura et al. teach the method according to claim 9 as noted above.
Zheng et al. do not explicitly teach all of wherein when generating a 3D contour of the examination, via a thermal image conversion matrix, the thermal image or the 2D contour is converted to be in the depth camera coordinate system, or the depth image is converted to be in the thermal camera coordinate system, wherein the thermal image conversion matrix is acquired via the following steps: positioning a calibration tool so that the calibration tool is in both a field of view of a depth camera and a field of view of a thermal camera; imaging the calibration tool via the depth camera and the thermal camera respectively, and calculating depth image interior angle coordinate values of an interior angle on the calibration tool in the depth camera coordinate system and thermal image interior angle coordinate values of the interior angle on the calibration tool in the thermal camera coordinate system, wherein the calibration tool is heated to generate a thermal difference from an original temperature thereof; and calculating the thermal image conversion matrix based on the depth image interior angle coordinate values and the thermal image interior angle coordinate values.
However, Nakamura et al. teach wherein when generating a 3D contour of the examination, via a thermal image conversion matrix, the thermal image or the 2D contour is converted to be in the depth camera coordinate system, or the depth image is converted to be in the thermal camera coordinate system, wherein the thermal image conversion matrix is acquired via the following steps: ("The calibration unit 551 saves data or information of the obtained projection conversion matrix in the memory 52 as a calibration result. Accordingly, the projection control device 50 can accurately convert the optional coordinates (Xc, Yc) in the infrared captured image IRG into the corresponding coordinates (Xp, Yp) in the projection image PJR and obtain the corresponding coordinates (Xp, Yp) by using the calibration result," par. 72) imaging the calibration tool via the depth camera and the thermal camera respectively, and calculating depth image interior angle coordinate values of an interior angle on the calibration tool in the depth camera coordinate system and thermal image interior angle coordinate values of the interior angle on the calibration tool in the thermal camera coordinate system, ("The calibration unit 551 performs a processing (that is, calibration) of obtaining a relational expression (for example, a projection conversion matrix) of a conversion processing that determines a correspondence relationship between a position (specifically, coordinates) in the infrared captured image generated by the IR camera 40 and a position (specifically, coordinates) in the projection image projected by the projector 10. Specifically, the calibration unit 551 detects where four corners of the projection image are positioned in the infrared captured image by designation using the input device IP of the user or by a predetermined image processing (for example, an edge detection processing), and obtains, for example, the above-described projection conversion matrix," par. 69) and calculating the thermal image conversion matrix based on the depth image interior angle coordinate values and the thermal image interior angle coordinate values ("The calibration unit 551 performs a processing (that is, calibration) of obtaining a relational expression (for example, a projection conversion matrix) of a conversion processing that determines a correspondence relationship between a position (specifically, coordinates) in the infrared captured image generated by the IR camera 40 and a position (specifically, coordinates) in the projection image projected by the projector 10. Specifically, the calibration unit 551 detects where four corners of the projection image are positioned in the infrared captured image by designation using the input device IP of the user or by a predetermined image processing (for example, an edge detection processing), and obtains, for example, the above-described projection conversion matrix," par. 69).
Additionally, Parrish et al. teach positioning a calibration tool so that the calibration tool is in both a field of view of a depth camera and a field of view of a thermal camera; ("A temperature-controlled calibration source 120 is positioned in a location within the FOVs of the thermal camera 106 and the visible camera 101," par. 49) and wherein the calibration tool is heated to generate a thermal difference from an original temperature thereof ("A temperature-controlled calibration source 120 is positioned in a location within the FOVs of the thermal camera 106 and the visible camera 101," par. 49).
Therefore, taking the teachings of Zheng et al., Prasad et al., Nakamura et al., and Parrish et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the imaging and 2D contour generating methods as taught by Zheng et al., the 3D contour generation methods and machine contact prediction methods as taught by Prasad et al., and the thermal image coordinate conversion methods as taught by Nakamura et al. to use the temperature and camera calibration techniques as taught by Parrish et al. The suggestion/motivation for doing so would have been that, “it may be convenient to make the temperature of the calibration source higher than the highest body temperature expected, as the calibration source will then likely be the hottest item in the FOV, and thus particularly easy to identify” as noted by the Nakamura et al. disclosure in paragraph [0050], which also motivates combination because the combination would predictably have a higher accuracy as there is a reasonable expectation that the combination achieves higher accuracy by improving temperature measurement precision and reliability, as the calibration source remains distinctly identifiable and serves as a reliable reference point across the field of view during thermal imaging and coordinate conversion; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Allowable Subject Matter
Claims 5, 6, 16, and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reasons for Indicating Allowable Subject Matter
The following is an examiner’s statement of reasons for allowance: The prior art of record does not teach certain distinguishing features as described below in reference to claim 5.
Regarding claim 5, the prior art Wang et al. teaches wherein the thermal contour image most conforming to the contour of the examination subject is selected
None teaches: via comparison with an a priori template image acquired in advance, wherein the a priori template image is acquired via the following steps: acquiring in advance a plurality of thermal images of different examination subjects under different conditions; performing segmentation processing on each of the plurality of thermal images separately based on a plurality of predetermined temperature thresholds to acquire a plurality of a priori thermal contour images, and selecting, from the plurality of a priori thermal contour images, an optimal thermal contour image most conforming to a contour of the examination subject; and extracting features from all the optimal thermal contour images corresponding to the plurality of thermal images, and creating the a priori template image based on the extracted features.
Further, none of the reference teaches or fairly suggests the combination of claimed elements. The Examiner finds no reason or motivation to combine the above references in an obviousness rejection thus placing the claim in condition for allowance.
Claim 16 is in condition for allowance to by analogy.
Claims 6 and 17 are in condition for allowance to as depending on an objected claim.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2024 0320834 A1 to Wang et al. discloses monitoring a patient's state during a medical procedure by using a camera to capture images of a body part via a reflection, finding the reflective surface's contour to set an image region, selecting pixels of the body part, and calculating the health signal from them.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KARSTEN F LANTZ whose telephone number is (571) 272-4564. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Karsten F. Lantz/Examiner, Art Unit 2664
Date: 8/19/2026
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights