DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 4 is objected to because of the following informalities:
Claim 4 is objected to on the basis of the limitation: “The method of claim 2, determining that the skimmer is present includes”. For clarification, Examiner recommends amending the claim as: “The method of claim 2, wherein determining that the skimmer is present includes”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 & 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6 & 7 recite the limitation "The method of claim 1, wherein determining that alignment is needed includes". There is insufficient antecedent basis for this limitation in the claims. For the purposes of examination, claims 6 and 7 are interpreted as dependent on claim 5, rather than claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process of comparing images) without significantly more.
Claim 1 recite(s):
“…generating a set of key points of the image”; Which can reasonably be interpreted as a human observer viewing an image and mentally generating a set of keypoints within the image.
“…comparing the aligned image to the baseline image to determine whether a skimmer is present”; Which can reasonably be interpreted as a human observer mentally comparing an aligned image to a baseline image to mentally determine if a skimmer is present.
“…in response to determining that the skimmer is present inside the card reader of the ATM, outputting an alert”; Which can reasonably be interpreted as a human observer verbally outputting an alert, in the case that a skimmer is determined to be present.
This judicial exception is not integrated into a practical application because of additional elements:
“capturing, using a camera, an image inside a card reader of an automated teller machine (ATM)”; is/are generically recited/well-understood, routine, and conventional extra-solution activity of data gathering.
“aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image”; is/are generically recited computer method(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a generically recited aligned image creation.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because of additional elements:
“capturing, using a camera, an image inside a card reader of an automated teller machine (ATM)”; is/are well-understood, routine, and conventional extra-solution activity of data gathering.
“aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image”; is/are well-understood, routine, and conventional computer method(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a well-understood, routine, and conventional aligned image creation.
Depending claims 2-9 do not remedy these deficiencies.
Claim(s) 2-4 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of comparing a region of interest in the aligned image to a correspond region of interest in the baseline image based on pixel values and a threshold. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 5-7 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of determining image alignment is needed by comparing the image to the baseline image, identifying the camera has moved, and determining whether the comparison triggers the alert. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 8 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an additional element of capturing a baseline image during an initial installation of the camera. The baseline image capturing is recited generically such that it does not provide any meaningful limitations on performing the abstract idea. This claim is not patent eligible.
Claim(s) 9 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an additional element of capturing the image in response to detection of a card’s removal. The image capturing is recited generically such that it does not provide any meaningful limitations on performing the abstract idea. This claim is not patent eligible.
Claim 10 recite(s):
“…generating a set of key points of the image”; Which can reasonably be interpreted as a human observer viewing an image and mentally generating a set of keypoints within the image.
“…comparing the aligned image to the baseline image to determine whether a skimmer is present”; Which can reasonably be interpreted as a human observer mentally comparing an aligned image to a baseline image to mentally determine if a skimmer is present.
“…outputting a result of the determination”; Which can reasonably be interpreted as a human observer verbally outputting a determination based on the image comparison.
This judicial exception is not integrated into a practical application because of additional elements:
“An automated teller machine (ATM) comprising: a card reader; a camera configured to capture an image of a portion of the card reader; processing circuitry; and memory including instructions,”; is/are generically recited computer element(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a generically recited automated teller machine (ATM).
“aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image”; is/are generically recited computer method(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a generically recited aligned image creation.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because of additional elements:
“An automated teller machine (ATM) comprising: a card reader; a camera configured to capture an image of a portion of the card reader; processing circuitry; and memory including instructions,”; is/are well-understood, routine, and conventional computer element(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a well-understood, routine, and conventional automated teller machine (ATM).
“aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image”; is/are well-understood, routine, and conventional computer method(s) that does/do not add a meaningful limitation to the abstract idea because it/they amount to simply implementing the abstract idea on a computer and pertain to a well-understood, routine, and conventional aligned image creation.
Depending claims 11-20 do not remedy these deficiencies.
Claim(s) 11 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of determining a skimmer is not present and logging the determination. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 12 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of outputting an alert in response to determining a skimmer is present. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 13-15 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of comparing a region of interest in the aligned image to a correspond region of interest in the baseline image based on pixel values and a threshold. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 16-18 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of determining image alignment is needed by comparing the image to the baseline image, identifying the camera has moved, and determining whether the comparison triggers the alert. This is a mental process. The claim(s) is/are not patent eligible.
Claim(s) 19 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an additional element of capturing a baseline image during an initial installation of the camera. The baseline image capturing is recited generically such that it does not provide any meaningful limitations on performing the abstract idea. This claim is not patent eligible.
Claim(s) 20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an additional element of capturing the image in response to detection of a card’s removal. The image capturing is recited generically such that it does not provide any meaningful limitations on performing the abstract idea. This claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8-10, 12, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Whytock (US-11790189-B1) in view of Xin et al. (US-20220036525-A1) hereinafter referenced as Xin.
Regarding claim 1, Whytock discloses: A method comprising: capturing, using a camera, an image inside a card reader of an automated teller machine (ATM) (Whytock: Col 1: Lines: 58-62: “Referring now to FIGS. 1 and 2, respective front and side views are depicted of a motorized card reader 10 that includes an internal camera 82 for use in detecting any foreign structure, such as a deep insert skimmer, inserted within the internal portions of card reader 10.”);
comparing the image to a baseline image to determine whether a skimmer is present; and
in response to determining that the skimmer is present inside the card reader of the ATM, outputting an alert (Whytock: Col 1: Lines 6-10: “This disclosure relates generally to a system and method for detecting the presence of a foreign object, such as a deep insert skimmer, placed internally within a card reader for a self-service terminal such as an automated teller machine (ATM).”;
Col 6: Lines 13-30: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60, at step 220…If the images are found to be different at step 230, processing proceeds to step 240, where an alert signal is generated indicating that a foreign object has been detected within card reader 10.”).
Whytock does not disclose expressly: generating a set of key points of the image;
aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image; comparing the aligned image to the baseline image to determine whether a skimmer is present.
Xin discloses: A method for analyzing visual defects in a test image based on registering and comparing the test image to a baseline, or template, image (Xin: Abstract). Wherein the method comprises: generating a set of key points of the test image; aligning key points of the set of key points of the test image to corresponding key points in a baseline image to create an aligned image (Xin: 0038: “At 206 , the image difference program 110 may perform an image registration between the test image and the template image. The image difference program 110 may determine a number of feature points in the test image. The image difference program 110 may determine a number of feature points in the template image…The image difference program 110 may use a method to determine accurate matching pairs of feature points between the test image and template image…The image difference program 110 may use the accurate matching pairs of feature points to generate a homography matrix.”;
0045-0046: “The image difference program 110 may use the homography matrix to map the test image to the template image. Mapping the test image to the template image may orientate the test image in a manner similar to the template image. Aligning the test image with the template image may reduce false defect detections.
At 208, the image difference program 110 may generate a registered test image. The registered test image is the test image orientated in a manner similar to (e.g., aligned) the template image. Each pixel of the registered test image may have a corresponding pixel in the template image.”); comparing the aligned image to the baseline image to determine differences between the images (Xin: 0025: “the present invention may include performing an image difference analysis between the template image and registered test image. Performing the image difference analysis with the registered test image as opposed to the original test image may reduce the rate of false defect detection. Additionally, if the image difference analysis determines that there is no material difference between the template image and registered test image the image difference process would end.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the algorithms for registering a test image based on a template image taught by Xin in order to register the captured image disclosed by Whytock to the reference image prior to comparing the images. The suggestion/motivation for doing so would have been “the present invention may decrease false detections by image registration. Image registration orientates the test image in a manner that aligns it with the template image, generating a registered test image. The registered test image is the test image orientated in a manner consistent with the template image. Frequently when a test image is captured it is orientated in a different manner than the template image because of camera instability. Camera instability may be random, making use of filtering difficult. Often an image difference analysis is performed between the test image and template image, often leading to false defect detections.” (Xin: 0024). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Whytock with Xin to obtain the invention as specified in claim 1.
Regarding claim 8, Whytock in view of Xin discloses: The method of claim 1, wherein the baseline image is captured by the camera during a process of initial installation of the camera in the ATM (Whytock: Col 5-6: Lines 50-10: “The cavity check performed at step 160 is done by comparing a reference image, generated during manufacturing for example, with a currently acquired image… The
reference image is generated during the manufacturing process or during installation of card reader 10 into a self-service terminal.”; Wherein the generating of the reference image during manufacturing constitutes the capturing of the image during an initial installation of the camera in the ATM.).
Regarding claim 9, Whytock in view of Xin discloses: The method of claim 1, wherein capturing the image includes capturing the image in response to detecting removal of a card from the card reader (Whytock: Col 5: Lines 34-49: “When the current transaction is completed, the controller 60 then ejects the bank card at step 140 using the linear transport mechanism 42 to present the bank card to the customer. The controller 60 ascertains if the bank card is removed by the customer at step 145, for example, by detecting closure of the shutter 26 using the shutter detect sensor 40…The controller 60 then enables the input read head 30 and external camera 80 at step 155 (to prepare for the next transaction) and performs a cavity check at step 160 to determine whether a foreign object (such as a deep insert skimmer) has been inserted into the card reader 10.”; Wherein the image capturing for skimmer detection is performed once the card is removed from the card reader.).
Regarding claim 10, Whytock discloses: An automated teller machine (ATM) comprising: a card reader; a camera configured to capture an image of a portion of the card reader (Whytock: Col 1: Lines: 58-62: “Referring now to FIGS. 1 and 2, respective front and side views are depicted of a motorized card reader 10 that includes an internal camera 82 for use in detecting any foreign structure, such as a deep insert skimmer, inserted within the internal portions of card reader 10.”);
processing circuitry; and memory including instructions (Whytock: Figure 4;Col 6: Lines 13-18: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60, at step 220.”; Wherein the image processing using an image processing module running on a controller implies the presence of processing circuitry executing instruction from memory.), which when executed by the processing circuitry, cause the processing circuitry to perform operations including:
comparing the image to a baseline image to determine whether a skimmer is present; and
outputting a result of the determination (Whytock: Col 1: Lines 6-10: “This disclosure relates generally to a system and method for detecting the presence of a foreign object, such as a deep insert skimmer, placed internally within a card reader for a self-service terminal such as an automated teller machine (ATM).”;
Col 6: Lines 13-30: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60, at step 220…If the images are found to be different at step 230, processing proceeds to step 240, where an alert signal is generated indicating that a foreign object has been detected within card reader 10.”).
Whytock does not disclose expressly: generating a set of key points of the image;
aligning key points of the set of key points of the image to corresponding key points in a baseline image to create an aligned image; and comparing the aligned image to the baseline image to determine whether a skimmer is present.
Xin discloses: A method for analyzing visual defects in a test image based on registering and comparing the test image to a baseline, or template, image (Xin: Abstract). Wherein the method comprises: generating a set of key points of the test image; aligning key points of the set of key points of the test image to corresponding key points in a baseline image to create an aligned image (Xin: 0038: “At 206 , the image difference program 110 may perform an image registration between the test image and the template image. The image difference program 110 may determine a number of feature points in the test image. The image difference program 110 may determine a number of feature points in the template image…The image difference program 110 may use a method to determine accurate matching pairs of feature points between the test image and template image…The image difference program 110 may use the accurate matching pairs of feature points to generate a homography matrix.”;
0045-0046: “The image difference program 110 may use the homography matrix to map the test image to the template image. Mapping the test image to the template image may orientate the test image in a manner similar to the template image. Aligning the test image with the template image may reduce false defect detections.
At 208, the image difference program 110 may generate a registered test image. The registered test image is the test image orientated in a manner similar to (e.g., aligned) the template image. Each pixel of the registered test image may have a corresponding pixel in the template image.”); and comparing the aligned image to the baseline image to determine differences between the images (Xin: 0025: “the present invention may include performing an image difference analysis between the template image and registered test image. Performing the image difference analysis with the registered test image as opposed to the original test image may reduce the rate of false defect detection. Additionally, if the image difference analysis determines that there is no material difference between the template image and registered test image the image difference process would end.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the algorithms for registering a test image based on a template image taught by Xin in order to register the captured image disclosed by Whytock to the reference image prior to comparing the images. The suggestion/motivation for doing so would have been “the present invention may decrease false detections by image registration. Image registration orientates the test image in a manner that aligns it with the template image, generating a registered test image. The registered test image is the test image orientated in a manner consistent with the template image. Frequently when a test image is captured it is orientated in a different manner than the template image because of camera instability. Camera instability may be random, making use of filtering difficult. Often an image difference analysis is performed between the test image and template image, often leading to false defect detections.” (Xin: 0024). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Whytock with Xin to obtain the invention as specified in claim 10.
Regarding claim 12, Whytock in view of Xin discloses: The ATM of claim 10, wherein outputting the result includes outputting an alert in response to determining that the skimmer is present inside the card reader of the ATM (Whytock: Col 1: Lines 6-10: “This disclosure relates generally to a system and method for detecting the presence of a foreign object, such as a deep insert skimmer, placed internally within a card reader for a self-service terminal such as an automated teller machine (ATM).”;
Col 6: Lines 13-30: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60, at step 220…If the images are found to be different at step 230, processing proceeds to step 240, where an alert signal is generated indicating that a foreign object has been detected within card reader 10.”).
As per claim(s) 19, arguments made in rejecting claim(s) 8 are analogous.
As per claim(s) 20, arguments made in rejecting claim(s) 9 are analogous.
Claim(s) 2-4 and 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Whytock in view of Xin, and further in view of Takeda et al. (US-20240054300-A1) hereinafter referenced as Takeda.
Regarding claim 2, Whytock in view of Xin discloses: The method of claim 1.
Whytock in view of Xin does not disclose expressly: wherein comparing the aligned image to the baseline image includes comparing a region of interest in the aligned image to a corresponding region of interest in the baseline image.
Takeda discloses: wherein comparing an image to a baseline image includes comparing a region of interest in the image to a corresponding region of interest in the baseline image (Takeda: 0066-0068: “In the step S19 , the particular region in which the hole having a particular shape exists is, for example, detected as a pixel range in the captured image which is obtained by the imaging device 84…Further, the host apparatus 10 may detect the particular region by excluding a region where a foreign matter such as an insert skimmer is unable to be entered and a region of an internal movable part of the card reader 100 from the captured image obtained by the imaging device 84…Next, the host apparatus 10 calculates a matching rate between the first captured image acquired in the step S 13 and the second captured image acquired in the step S 18 in the particular region which has been detected in the step S 19 (step S 20 ). Specifically, the host apparatus 10 calculates a matching rate based on comparison of pixel values between the particular region of the first captured image and the particular region of the second captured image.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the algorithms for determining and comparing the pixels of regions in the first and second captured images taught by Takeda in order to compare the registered and reference images disclosed by Whytock in view of Xin. The suggestion/motivation for doing so would have been “image analysis is performed in a limited region of the hole which is available to fixing of the magnetic data swindle member and thus, influence of a region where likelihood that a magnetic data swindle member exists is low is reduced and the magnetic data swindle member can be detected with a high degree of accuracy.” (Takeda: 0096). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Whytock in view of Xin with Takeda to obtain the invention as specified in claim 2.
Regarding claim 3, Whytock in view of Xin and Takeda discloses: The method of claim 2, wherein determining that the skimmer is present includes determining that the region of interest differs from the corresponding region of interest by greater than a threshold (Takeda: 0070-0072: “the host apparatus 10 may divide a captured image (first captured image and second captured image) obtained by the imaging device 84 into a plurality of cells (block) in an array shape or a matrix shape, and the matching rate is calculated in each of the cells overlapped with the particular region among the respective divided cells…in a case that a matching rate of each of the cells is calculated, the host apparatus 10 may determine whether a foreign matter exists in the space 90 or not based on the number of the cells whose matching rate is not less than the threshold, distribution (adjacent number) of the cells whose matching rate is not less than the threshold, or the like.”).
Regarding claim 4, Whytock in view of Xin and Takeda discloses: The method of claim 2, wherein determining that the skimmer is present includes determining that pixel values in the region of interest differ from corresponding pixel values in the corresponding region of interest (Takeda: 0069: “the host apparatus 10 determines to be “matching” when a difference between a pixel value of the first captured image and a pixel value of the second captured image among respective pixels of the particular region is less than a threshold and, in a case that a difference between a pixel value of the first captured image and a pixel value of the second captured image is equal to or more than the threshold, the host apparatus 10 determines to be “not matching”. Further, the host apparatus 10 calculates a ratio of the pixels determined to be “matching” among the respective pixels of the particular region as a matching rate (matching degree).”).
Regarding claim 11, Whytock in view of Xin discloses: The ATM of claim 10, wherein the result of the determination is that no skimmer is present (Whytock: Col 6: Lines 13-26: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60, at step 220. A determination is made, at step 230, whether the reference image is different from acquired image based on the comparison made in step 220 . If the images are not different, processing proceeds to step 260 which ends the currently performed cavity check processing. If the images are found to be different at step 230 , processing proceeds to step 240 , where an alert signal is generated indicating that a foreign object has been detected within card reader 10.”).
Whytock in view of Xin does not disclose expressly: wherein outputting the result includes saving an indication of the result in a log.
Takeda discloses: wherein outputting a result includes saving an indication of the result in a log (Takeda: 0083: “Further, when respective captured images obtained by the imaging device 84 are stored in a memory, the captured images can be utilized as an image log which indicates at which time and what kind of abnormality has occurred.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image log taught by Takeda in order to store the captured images and foreign object detection results disclosed by Whytock in view of Xin. The suggestion/motivation for doing so would have been “when respective captured images obtained by the imaging device 84 are stored in a memory, the captured images can be utilized as an image log which indicates at which time and what kind of abnormality has occurred.” (Takeda: 0083; Wherein stored images serve to indicate timestamped detection results along with the image processed). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Whytock in view of Xin with Takeda to obtain the invention as specified in claim 11.
As per claim(s) 13, arguments made in rejecting claim(s) 2 are analogous.
As per claim(s) 14, arguments made in rejecting claim(s) 3 are analogous.
As per claim(s) 15, arguments made in rejecting claim(s) 4 are analogous.
Claim(s) 5-7 and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Whytock in view of Xin, and further in view of Shen et al. (CN-106880906-A) hereinafter referenced as Shen.
Regarding claim 5, Whytock in view of Xin discloses: The method of claim 1.
Whytock in view of Xin does not disclose expressly: further comprising before generating the set of key points of the image, comparing the image to the baseline image to determine whether alignment is needed.
Shen discloses: A method for verifying patient positioning during radiotherapy based on registering a captured image to a baseline, or reference image (Shen: 0008: “The purpose of this invention is to provide a real-time verification device and method for patient positioning during radiotherapy, addressing the shortcomings of existing technologies.”;
0012: “(1) Initial patient positioning and acquisition of standard reference images: When the patient receives treatment for the first time in the treatment room, the three cameras are connected according to the verification device settings, and the control software on the graphics workstation of the verification device is turned on; the patient is positioned according to the patient positioning parameters output by the treatment plan system, and the correct patient positioning is determined by registering the images acquired by the conventional DR equipment with the DRR images output by the treatment plan system through conventional positioning verification methods. The standard reference positioning optical image of each camera is acquired using the algorithm software on the image workstation, and the optical image is stored in the database.”). Wherein the method comprises, before generating a set of key points of the captured image, comparing the captured image to the baseline image to determine whether alignment is needed (Shen: 0013-0015: “(2) Acquisition of difference image: When the patient is treated in subsequent sessions, the difference between the real-time video image captured by each camera and the standard reference image of that camera is calculated…
(3) Obtaining the region of interest: The image of the positioning deviation is first smoothed by a mean filter with a 5*5 kernel and a Gaussian filter with a 7*7 kernel. The smoothed difference image is then extracted using the Canny operator to obtain the coordinates of the largest rectangular region containing all contours, i.e., the coordinates of the region of interest…
(4) Patient positioning verification: Calculate the corner points on the standard reference image and the real-time video image within the coordinates of the region of interest, and match the corner points of the two images; obtain the coordinate deviation value of the matched corner points. If the deviation value is less than the set threshold of 1 mm, the patient positioning is considered correct. If the deviation value is greater than 1 mm, the patient is repositioned according to the deviation value”;
0016: “(5) Obtaining patient positioning deviation: Map the obtained ROI region coordinates onto the reference image and the real-time video image. Extract corner points from the mapped region using the improved ORB (Oriented Fast and Rotated BRIEF) corner point extraction and registration algorithm, and register the corner points in the ROI region on the two images;”;
Wherein if the images are aligned, using ORB, if the differences between the images is larger than a threshold.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the known technique of comparing the reference and video images differences to a threshold, prior to aligning the images, taught by Shen into the foreign object detection method disclosed by Whytock in view of Xin by comparing the reference and captured images prior to performing alignment. The suggestion/motivation for doing so would have been “The real-time verification device and method for patient positioning in radiotherapy are designed to use multi-channel ordinary optical cameras to perform rapid and accurate patient positioning verification, and to monitor the patient's position status in real time during treatment…Its advantages also include: (1) It is inexpensive and has high imaging accuracy; (2) The optical camera imaging does not add any extra radiation dose to the patient, effectively reducing the risk of complications caused by extra radiation dose during treatment; (3) The improved algorithm increases the computing speed, which can quickly guide and verify the patient's online positioning” (Shen: 0020; Wherein the verification prior to performing alignment allows for reduction in processing time, increasing speed, if alignment is not needed.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Whytock in view of Xin with Shen to obtain the invention as specified in claim 5.
Regarding claim 6, Whytock in view of Xin and Shen discloses: The method of claim 1 (Claim limitation is interpreted according to the rejection of claim 6 under 35 U.S.C. 112(b) disclosed above, i.e., as dependent on claim 5.), wherein determining that alignment is needed includes identifying that the camera has moved (Xin: 0024: “the present invention may decrease false detections by image registration. Image registration orientates the test image in a manner that aligns it with the template image, generating a registered test image. The registered test image is the test image orientated in a manner consistent with the template image. Frequently when a test image is captured it is orientated in a different manner than the template image because of camera instability. Camera instability may be random, making use of filtering difficult. Often an image difference analysis is performed between the test image and template image, often leading to false defect detections. Image registration of the test image transforms the test image into a manner more consistent with the template image. Image registration utilizes distinct points (e.g., corner features and gradient features) to determine good matching points (e.g., interest points) between the test image and template image.”; Wherein the need for image registration is determined as camera instability.).
Regarding claim 7, Whytock in view of Xin and Shen discloses: The method of claim 1 (Claim limitation is interpreted according to the rejection of claim 7 under 35 U.S.C. 112(b) disclosed above, i.e., as dependent on claim 5.), wherein determining that alignment is needed includes determining that the comparison of the image to the baseline image triggers the alert (Shen: 0015: “(4) Patient positioning verification: Calculate the corner points on the standard reference image and the real-time video image within the coordinates of the region of interest, and match the corner points of the two images; obtain the coordinate deviation value of the matched corner points. If the deviation value is less than the set threshold of 1 mm, the patient positioning is considered correct. If the deviation value is greater than 1 mm, the patient is repositioned according to the deviation value.”)
(Whytock: Col 6: Lines 13-26: “The steps performed in the cavity check are shown in the flowchart 200 of FIG. 6. First, at step 210, an image is acquired, via internal camera 82, of an internal portion of the card reader 10. This image is compared with a reference image, using for example an image processing module running on controller 60 , at step 220 . A determination is made, at step 230 , whether the reference image is different from acquired image based on the comparison made in step 220 . If the images are not different, processing proceeds to step 260 which ends the currently performed cavity check processing. If the images are found to be different at step 230 , processing proceeds to step 240 , where an alert signal is generated indicating that a foreign object has been detected within card reader 10.”; Wherein if the images are found to be different, an alert is sounded.).
As per claim(s) 16, arguments made in rejecting claim(s) 5 are analogous.
As per claim(s) 17, arguments made in rejecting claim(s) 6 are analogous.
As per claim(s) 18, arguments made in rejecting claim(s) 7 are analogous.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights