Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1, 6, 8, 12, 17, 19, and 22 have been amended. Claims 2-5, 9-10, 13-16, and 20-21 have been canceled. Claims 23-24 have been added. Claims 1, 6-8, 11-12, 17-19, and 22-24 are pending.
Response to Arguments
The standing 101 issue shave been corrected by way of amendment, specifically, how to claim as a whole integrates into a practical application as described on page 9 of the remarks.
Regarding the remarks filed 09/22/26, on pages 9-11, applicant asserts:
“Yamamoto does not contain any description regarding a hardware processor that "analyzes the acquired two-dimensional radiation image to infer three-dimensional structure information including near side structure information indicating the near side structure and far side structure information indicating the far side structure,"and "displays, on a display part, a screen including the re-imaging support information and the three- dimensional structure information.”(Page 11)
The examiner respectfully disagrees in part. YAMAMOTO does teach a hardware processor that performs “analyzing the acquired two-dimensional radiation image to infer three-dimensional structure information including near side structure information indicating the near side structure and far side structure information indicating the far side structure” (Fig. 8, [0090]: “That is, as shown in FIGS. 8 and 9, the controller 41 is configured to perform control to find the outer edge Bi of the medial condyle based on analysis performed on the pre-captured X-ray image Gx1 acquired through the learned model Md1. The controller 41 is configured to perform control to find the outer edge Bo of the lateral condyle based on analysis through the learned model Md2 acquired”. The controller acts as the hardware processor which controls the models to analyze the images to obtain the three-dimensional structure information. The outer edge of the medial condyle and the lateral condyle are used to make up the inferred three-dimensional structure information, this structure information is on the inside of the subject (bones). Dotted and solid lines are used to infer the line opposite of the scanner and the line towards the scanner so depth (a third dimension) is inferred “Here, in FIG. 9, the identification marks Ma are a thick solid line for emphasizing the outer edge Bi of the medial condyle and a thick dotted line for emphasizing the outer edge Bo of the lateral condyle” [0090]. This structure on the inside of a subject includes a near side structure and a far side structure in an irradiation direction “Specifically, the controller 41 is configured to perform control to find, based on the pre-capture X-ray image Gx1, the outer edge Bi of the medial condyle, which is a part of the outer edge on the X-ray irradiator 11 side in the irradiation direction included in the bone of the patient Pa in the pre-capture X-ray image Gx1, and the outer edge Bo of the lateral condyle, which is a part of the outer edge on a side opposite to the X-ray irradiator 11 side included in the bone of the patient, to acquire the position correction information J1” [0089]).
Therefore, YAMAMOTO does teach the analyzing step. As for the displaying step, YAMAMOTO does not expressly disclose displaying the three-dimensional structure information alongside the re-imaging support information.
Applicant further asserts:
“Sugiyama is cited in regards to its teaching of a display control unit 50 that displays an X-ray image with an enlarged image of the region of interest ROI. Sugiyama does not teach or suggest a hardware processor that "analyzes the acquired two-dimensional radiation image to infer three-dimensional structure information including near side structure information indicating the near side structure and far side structure information indicating the far side structure," and "displays, on a display part, a screen including the re-imaging support information and the three-dimensional structure information”(Page 12)
The examiner respectfully disagrees in part. As seen in the mapping above, YAMAMOTO teaches the analyzing step, so SUGIYAMA does not need to teach that limitation, as for the displaying step missing from YAMAMOTO, SUGIYAMA does teach a hardware processor that performs displaying the three-dimensional structure information alongside the re-imaging support information ([0099]: “In the above embodiment, the presentation unit 54 displays the enlarged image 70 of the region of interest ROI that the determination unit 52 pays attention to in the image display region 60A (refer to FIG. 9). As shown in FIG. 12, the presentation unit 54 may display a pixel region 71, which has contributed to the determination of the reason for reimaging by the determination unit 52, in the enlarged image 70”. The three dimensional structure information (70 and 71) is displayed alongside the re-imaging support information (64). The presentation unit (54) is part of a hardware processor (CPU 32)).
Therefore SUGIYAMA does teach the displaying step and does not need to teach the analyzing step.
Since YAMAMOTO teaches the analyzing step using a hardware processor, and SUGIYAMA teaches the displaying step using a hardware processor, the combination of YAMAMOTO and SUGIYAMA teaches using hardware processors to perform the steps of analyzing and displaying. Therefore the claims stand rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-8, 11-12, 17-19, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over YAMAMOTO et al. (US 20250387089 A1 Hereinafter “YAMAMOTO”) in view of SUGIYAMA (US 20230015883 A1 Hereinafter “SUGIYAMA”).
Regarding claim 1, YAMAMOTO teaches an image processing apparatus comprising:
a hardware processor ([0064]: “The controller 21 includes a CPU, a GPU (Graphics Processing Unit) and the like”),
wherein the hardware processor acquires a two-dimensional radiation image of a structure on an inside of a subject, the structure including a near side structure and a far side structure in an irradiation direction (Fig. 13, [0101]: “As shown in FIG. 13, the mobile terminal 4 acquires the pre-captured X-ray image Gx1, which is captured through X-rays with which the patient Pa is irradiated from the X-ray irradiator 11 and that pass through the patient, in step S1”. This image includes the structure on the inside of a subject, the structure including a near side structure and a far side structure in an irradiation direction “Specifically, the controller 41 is configured to perform control to find, based on the pre-capture X-ray image Gx1, the outer edge Bi of the medial condyle, which is a part of the outer edge on the X-ray irradiator 11 side in the irradiation direction included in the bone of the patient Pa in the pre-capture X-ray image Gx1, and the outer edge Bo of the lateral condyle, which is a part of the outer edge on a side opposite to the X-ray irradiator 11 side included in the bone of the patient, to acquire the position correction information J1”[0089]. These positions in an irradiation direction are represented by dotted and solid lines are used to infer the line opposite of the scanner (far side) and the line towards the scanner (near side)“Here, in FIG. 9, the identification marks Ma are a thick solid line for emphasizing the outer edge Bi of the medial condyle and a thick dotted line for emphasizing the outer edge Bo of the lateral condyle” [0090]. If these structures are identified, they must be in the image acquired); and
analyzes the acquired two-dimensional radiation image to infer three-dimensional structure information including near side structure information indicating the near side structure and far side structure information indicating the far side structure (Fig. 8, [0090]: “That is, as shown in FIGS. 8 and 9, the controller 41 is configured to perform control to find the outer edge Bi of the medial condyle based on analysis performed on the pre-captured X-ray image Gx1 acquired through the learned model Md1. The controller 41 is configured to perform control to find the outer edge Bo of the lateral condyle based on analysis through the learned model Md2 acquired”. The outer edge of the medial condyle and the lateral condyle are used to make up the inferred three-dimensional structure information, this structure information is on the inside of the subject (bones). Dotted and solid lines are used to infer the line opposite of the scanner and the line towards the scanner so depth (a third dimension) is inferred “Here, in FIG. 9, the identification marks Ma are a thick solid line for emphasizing the outer edge Bi of the medial condyle and a thick dotted line for emphasizing the outer edge Bo of the lateral condyle” [0090]. This image includes the structure on the inside of a subject, the structure including a near side structure and a far side structure in an irradiation direction “Specifically, the controller 41 is configured to perform control to find, based on the pre-capture X-ray image Gx1, the outer edge Bi of the medial condyle, which is a part of the outer edge on the X-ray irradiator 11 side in the irradiation direction included in the bone of the patient Pa in the pre-capture X-ray image Gx1, and the outer edge Bo of the lateral condyle, which is a part of the outer edge on a side opposite to the X-ray irradiator 11 side included in the bone of the patient, to acquire the position correction information J1” [0089]);
determines positioning misalignment of the subject during imaging of the two- dimensional radiation image based on the near side structure information and the far side structure information ([0093]: “Specifically, the controller 41 is configured to control to search for the positions of the outer edge Bi of the medial condyle and the outer edge Bo of the lateral condyle that bring the medial condyle and the lateral condyle that are found to substantially agree with each other by moving one of the medial condyle and the lateral condyle. Here, the positions to be searched are positions that minimize an area formed by the deviation between the outer edge Bi of the medial condyle and the outer edge Bo of the lateral condyle after the moving. Accordingly, the deviation between the outer edge Bi of the medial condyle and the outer edge Bo of the lateral condyle is found by using the deviation amount and the deviation direction”. The deviation between the deviation between the outer edge Bi of the medial condyle and the outer edge Bo of the lateral condyle is the misalignment based on the near side structure information and the far side structure information);
determines a positioning correction direction based on the positioning misalignment of the subject to generate re-imaging support information including the positioning correction direction, the positioning correction direction being a direction in which positioning of the subject or an imaging device is to be corrected at a time of re-imaging ([0094]: “The controller 41 is configured to perform control to calculate, based on the deviation amount and the deviation direction found, the position correction information J1 for correcting the relative position of the X-ray irradiator 11 relative to the knee of the patient Pa to a position at which the main captured X-ray image Gx2 (see FIG. 6) can be captured with the outer edge Bi of the medial condyle and the outer edge Bo of the lateral condyle overlapping each other when capturing an image of the knee of the patient Pa. That is, the controller 41 is configured to perform control to calculate the movement direction θ and the movement distance Dt of the X-ray irradiator 11 as the position correction information J1 based on the deviation amount and the deviation direction”. The position correction information encompasses a positioning correction direction be corrected at a time of re-imaging for the imaging device); and
displays, on a display part, a screen including the re-imaging support information ([0105]: “In step S5, the mobile terminal 4 displays the position correction information J1 on the display 44 based on the acquisition of the position correction information J1. Specifically, the mobile terminal 4 displays an arrow on the display 44 as the position correction information J1, which includes information on the movement direction θ in horizontal directions and the movement distance Dt when the radiologic technologist Ra manually moves the X-ray irradiator 11 and the patient Pa relative to each other. Here, the base end of the arrow (see FIG. 11)”. Pf is the corrected position in Fig. 11, and J1 is the position correction information needed to change the current capture state to the corrected one).
YAMAMOTO does not expressly disclose displaying, on a display part, a screen including the re-imaging support information and the three-dimensional structure information.
However, SUGIYAMA teaches displaying, on a display part, a screen including the re-imaging support information and the three-dimensional structure information ([0099]: “In the above embodiment, the presentation unit 54 displays the enlarged image 70 of the region of interest ROI that the determination unit 52 pays attention to in the image display region 60A (refer to FIG. 9). As shown in FIG. 12, the presentation unit 54 may display a pixel region 71, which has contributed to the determination of the reason for reimaging by the determination unit 52, in the enlarged image 70”. The three dimensional structure information (70 and 71) is displayed alongside the re-imaging support information (64). The presentation unit (54) is part of a hardware processor (CPU 32)).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify YAMAMOTO’s re-imaging determination method to include SUGIYAMA’s hardware processor’s ability to perform displaying the three dimensional structure information alongside the re-imaging support information because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify YAMAMOTO to include SUGIYAMA is expressly provided by SUGIYAMA, stating that by displaying the information in this way, it can help the technician more accurately determine the necessity of reimaging ([0099]: “As a result, the technician RG can more accurately determine the necessity of reimaging”). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify YAMAMOTO’s re-imaging determination method to include SUGIYAMA’s hardware processor’s ability to perform displaying the three dimensional structure information alongside the re-imaging support information with the motivation of improving re-imaging determination accuracy. The person of ordinary skill in the art would have recognized the benefit of improved re-imaging determination accuracy.
Regarding claim 6, the combination of YAMAMOTO and SUGIYAMA teaches the image processing apparatus according to claim 1, in addition, YAMAMOTO further teaches,
wherein the hardware processor acquires imaging site information ([0073]: “The learned model Md1 and the learned model Md2 are exemplary learned models that are to be used when capturing images of a medial condyle of a femur (see FIG. 5) and a lateral condyle of the femur (see FIG. 5) in a knee joint of the patient Pa. The learned model Md3 is a learned model that is to be used when capturing an elbow joint of patient Pa, for example”. For the controller to know which model to use, imaging site information has to be obtained),
wherein the hardware processor extracts a structure of the subject from the two-dimensional radiation image based on the acquired imaging site information (Fig. 8, [0090]: “That is, as shown in FIGS. 8 and 9, the controller 41 is configured to perform control to find the outer edge Bi of the medial condyle based on analysis performed on the pre-captured X-ray image Gx1 acquired through the learned model Md1. The controller 41 is configured to perform control to find the outer edge Bo of the lateral condyle based on analysis through the learned model Md2 acquired”. Since the models are used, the extraction of subject structure is based on the imaging site information).
Regarding claim 7, the combination of YAMAMOTO and SUGIYAMA teaches the image processing apparatus according to claim 1, in addition, YAMAMOTO further teaches,
wherein the three-dimensional structure information inferred by the hardware processor includes information of a medial condyle side line and information of a lateral condyle side line of a femoral condyle ([0073]: “The learned model Md1 and the learned model Md2 are exemplary learned models that are to be used when capturing images of a medial condyle of a femur (see FIG. 5) and a lateral condyle of the femur (see FIG. 5) in a knee joint of the patient Pa. The learned model Md3 is a learned model that is to be used when capturing an elbow joint of patient Pa, for example”. The medial and lateral condyle are mart of the femur, the femoral condyle is the condyle of the femur, which is the information being inferred).
Regarding claim 8, the combination of YAMAMOTO and SUGIYAMA teaches the image processing apparatus according to claim 1, in addition YAMAMOTO further teaches
wherein the hardware processor displays the inferred three-dimensional structure information (Fig. 9 shows the controller displaying the 3D structure information to the radiologist).
SUGIYAMA further teaches superimposing information for re-imaging on a 2D radiation image ([0099]: “In the above embodiment, the presentation unit 54 displays the enlarged image 70 of the region of interest ROI that the determination unit 52 pays attention to in the image display region 60A (refer to FIG. 9). As shown in FIG. 12, the presentation unit 54 may display a pixel region 71, which has contributed to the determination of the reason for reimaging by the determination unit 52, in the enlarged image 70”).
At the time the invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify YAMAMOTO’s re-imaging determination method to include SUGIYAMA’s superimposing of data that determines re-imaging on two-dimensional radiation images because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify YAMAMOTO to include SUGIYAMA is expressly provided by SUGIYAMA, stating that by displaying the information in this way, it can help the technician more accurately determine the necessity of reimaging ([0099]: “As a result, the technician RG can more accurately determine the necessity of reimaging”). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify YAMAMOTO’s re-imaging determination method to include SUGIYAMA’s superimposing of data that determines re-imaging on two-dimensional radiation images with the motivation of improving re-imaging determination accuracy. The person of ordinary skill in the art would have recognized the benefit of improved re-imaging determination accuracy.
Regarding claim 11, the combination of YAMAMOTO and SUGIYAMA teaches the image processing apparatus according to claim 1, in addition, YAMAMOTO further teaches,
wherein the hardware processor inputs, in a machine learning model which has learned radiation images including a structure inside a subject as input data and three-dimensional structure information on the structure inside the subject as correct output data, the radiation image acquired by the hardware processor to infer the three-dimensional structure information of the structure in the radiation image ([0072]: “The learned model Md1, the learned model Md2, the learned model Md3 are learned models that have been produced by machine learning using X-ray images Gx of imaging target parts as input data. Here, the imaging target parts include joint parts such as shoulders and elbows in artificial bones or bones, for example. Also, the learned model Md1, the learned model Md2, the learned model Md3 are models acquired based on machine learning such as U-Net, for example. Here, the machine learning using X-ray images Gx as input data can be any of supervised learning, unsupervised learning, and reinforcement learning”. The learned models are machine models which have been trained using radiation images (to be able to decipher structure in radiation images) of the structure inside a subject, and they produce three-dimensional structure information as output data).
Regarding claim 12, the content of claim 12 is similar to the content of claim 1, with the additional teachings of a non-transitory computer readable storage medium. YAMAMOTO also discloses this information [0064]: “The storage 22 includes a non-volatile storage medium such as HDD (Hard Disk Drive) or SSD (Solid State Drive)”. The non-volatile storage medium stores the program information to cause a computer to function as all the listed sections in claim 12). Therefore, claim 12 is rejected for the same reasons of obviousness as claim 1, along with the additional teachings above.
Regarding claim 17, the content of claim 17 is similar to the content of claim 6, therefore it is rejected for the same reasons of obviousness as claim 6.
Regarding claim 18, the content of claim 18 is similar to the content of claim 7, therefore it is rejected for the same reasons of obviousness as claim 7.
Regarding claim 19, the content of claim 19 is similar to the content of claim 8, therefore it is rejected for the same reasons of obviousness as claim 8.
Regarding claim 22, the content of claim 22 is similar to the content of claim 1, therefore it is rejected for the same reasons of obviousness as claim 1.
Allowable Subject Matter
Claims 23-24 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Landon et al. (US 11259874 B1) teaches machine learning for bone detection
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEFANO A DARDANO whose telephone number is (703)756-4543. The examiner can normally be reached Monday - Friday 11:00 - 7:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Greg Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEFANO ANTHONY DARDANO/ Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
