DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-14 were pending in the application filed November 22, 2023. Claims 1, 3-6, 8-14, are amended, no claims cancelled, and no additional claims added as of the remarks and amendments received March 2, 2026. Accordingly, claims 1-14 remain pending in the application for examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-14 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe (US 2022/0148323), and in view of Shapiro et al. (hereafter referred to as ‘Shapiro’, US 6,246,782).
Regarding claim 1 (Currently Amended), Watanabe discloses a teaching data generation device (Figs. 2,3&6) comprising:
an input interface that acquires at least one input image including an image of a recognition target (Fig. 3, input image G10-1 includes an image of a recognition target R10);
at least one processor that performs
a first process to generate polygon data along an outline of a portion in a first region of the at least one input image, the portion determined to be the image of the recognition target (Fig. 3 and pg. [0107]-[0108], [0117], polygon data “T1” is generated that indicates the outline of a region enclosing the target R10),
a second process to set segments in the at least one input image based on a luminance gradient (pg. [0111]-[0112] the “second region information … determined so that likelihood (reliability) as the contour of the initial region becomes higher”; pg. [0186]-[0187] “device 10 calculates a difference in brightness near the boundary or a gradient of change”. Generating the “second region information” sets the input image into at least two segments: the target region segment and the non-target/background region segment),
generation of modified polygon data which modify the polygon data based on the segments set in the second process (pg. [0115]-[0116] correcting the contour based on the second region information. Fig. 3 shows a modified polygon data T2 generated based on T1 and the second region information), and
generation of teaching data by adding label information to the at least one input image; and
an output interface that outputs the teaching data (pg. [0069]-[0070], [0083] “region information of the target region is used as teacher data for machine learning”; “a label attached to the target region will also be used as teacher data for machine learning”. Fig. 2, the generated teaching data is output to the learning device 50).
Watanabe fails to expressly disclose that “super pixel is performed to divide the at least one input image into the segments in the second process”.
In the same field of segmenting target object using machine learning, Shapiro discloses (Fig. 2) generating super pixels (step 24 and col. 5, lines 25-35) that are input into a neural net (step 25).
It is well known in the art that superpixels can represent images with fewer primitives, making neural networks more efficient and reliable. Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to which the claimed invention pertains to combine the teachings of Shapiro with that of Watanabe to yield the invention as described in claim 1. This combination (modification) could be made using known methods with no changes to the operating principles of either reference to produce nothing more than highly predictable results.
Regarding claim 2 (Previously Presented), Watanabe discloses the teaching data generation device according to claim 1, wherein the generation of the polygon data is performed based on recognition of foreground and background included in the at least one input image in the first process (Fig. 3, polygon T1 is generated based on recognition of target R10).
Even though Watanabe discloses (pg. [0101]) that “machine learning may be used for the segmentation algorithm”, Watanabe fails to expressly disclose that “super pixel is performed to divide the at least one input image into the segments in the second process”.
In the same field of segmenting target object using machine learning, Shapiro discloses (Fig. 2) generating super pixels (step 24 and col. 5, lines 25-35) that are input into a neural net (step 25).
It is well known in the art that superpixels can represent images with fewer primitives, making neural networks more efficient and reliable. Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to which the claimed invention pertains to combine the teachings of Shapiro with that of Watanabe to yield the invention as described in claim 2. This combination (modification) could be made using known methods with no changes to the operating principles of either reference to produce nothing more than highly predictable results.
Regarding claim 3 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 1, wherein the at least one processor performs the second process in a second region which includes at least one portions of the at least one input image, each of at least one portions determined to be the image of the recognition target, and the addition of the label information to the at least one portions (Fig. 3, each repetition of fitting updates the boundary. For example, T2 corresponds to the claimed second region. Annotation data is generated thereafter, see Fig. 6).
Regarding claim 4 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 3, wherein the at least one processor performs the second process for the portion where the polygon data is generated in the first process input image (Figs. 3&6, pg. [0107], T2 is generated based on T1).
Regarding claim 5 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 3, wherein the at least one processor sets the second region so as to be smaller than the first region (Fig. 3, T2 encloses a smaller region than T1 does).
Regarding claim 6 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 1, wherein the at least one processor sets the segments by performing the second process for a region including the recognition target in the at least one input image and generates the polygon data after the second process has been performed (pg. [0115]-[0116] correcting the contour based on the second region information. Fig. 3 shows a modified polygon data T2 generated based on T1 and the second region information)
Regarding claim 7 (Previously Presented), Watanabe discloses the teaching data generation device according to claim 1, wherein the label information is added to the at least one input image to extract the image of the recognition target matched with the label information (pg. [0069]-[0070], [0083], see analysis of claim 1).
Regarding claim 8 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 1, wherein the at least one processor generates the polygon data in the first process based on a certain algorithm including at least one selected from the group consisting of background removal based on hue information, graph cut, and detection of the recognition target by machine learning model (Fig. 6, S12, pg. [0101] “perform fitting using a segmentation algorithm by graph cut … Alternatively, machine learning may be used for the segmentation algorithm”).
Regarding claim 9 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 8, wherein the at least one processor performs the graph cut so as to cut the outline of the recognition target from the at least one input image based on a cost function (pg. [0112] “by using the graph cut using a random field that is set on the basis of the first region information (information indicating the boundary) as a cost function”).
Regarding claim 10 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 8, wherein the at least one processor performs training of the second machine learning model based on a result of the modification of the polygon data based on the segments set in the second process (pg. [0069]-[0070], [0083] “region information of the target region is used as teacher data for machine learning”; “a label attached to the target region will also be used as teacher data for machine learning”. Fig. 2, the generated teaching data is output to the learning device 50).
Regarding claim 11, Watanabe discloses the teaching data generation device according to claim 1, wherein the at least one processor corrects the at least one input image based on a difference between the polygon data and the modified polygon data or based on the modification of polygon data (Fig. 3, input image G10 is corrected by modification of polygon T).
Regarding claim 12 (Currently Amended), Watanabe discloses the teaching data generation device according to claim 1, but fails to expressly disclose wherein the at least one processor modifies a preprocessing parameter value applied to preprocessing to the at least one input image that is to be processed later of the polygon data.
However, preprocessing raw image data is well known and common practice in medical image processing, as for example disclosed in Shapiro (col. 4, lines 11-32, “reduces the scale of the image to a resolution of approximately 230 microns. This reduces the processing requirement while maintaining sufficient resolution to identify regions of interest (ROis). A blurring/contrast reduction algorithm is used …”). Resolution is parameter.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to which the claimed invention pertains to preprocess raw medical image data, as is common in medical image processing, to yield the invention as described in claim 12.
Claims 13 and 14 have been analyzed and are rejected for the same reasons as outlined above in the rejection of claim 1.
Response to Arguments
Applicant's arguments filed March 2, 2026 have been thoughtfully considered but they are not persuasive.
The Examiner most respectfully disagrees with Applicants assertion that Shapiro's superpixels are used as input to a neural network for image analysis, not to set segments based on a luminance gradient that are then used to modify polygon data as required by amended claim 1 from which claims 2 and 12 depend. Shapiro teaches:
PNG
media_image1.png
724
466
media_image1.png
Greyscale
(23) 3. Analyze Regions of Interest.
(24) Each region of interest is analyzed by means of a neural network using a super-pixeled image as an input. This grid is a radial-polar grid with the angles evenly spaced and constant radial increments. In the preferred embodiment, the grid consists of 10 equally-spaced radial and 32 equally-spaced angular bands. The pixels in each grid space are averaged together to create 320 "super-pixels" (reference numeral 24 in FIG. 2). The super-pixels, along with two inputs determined from the approximate size of the feature, are used as the inputs to the first neural net 14. For example, FIG. 4 is a section of a mammogram corresponding to FIG. 3 that has been overlaid with a super-pixel grid. FIG. 5 is a section of a mammogram corresponding to FIG. 3 after the region of interest has been reduced to super-pixels.
(25) The first neural network 14 is trained on both cancerous and non-cancerous regions of interest. Regions of Interest (ROIs) are selected from known (verified normal or verified cancerous) mammograms. These ROIs are ranked according to their appearance, with +1 being most normal-like and -1 being most lesion-like. The ROIs are then converted to super-pixels for input to the first neural network. A random scaling, rotation and translation of the super-pixel grid is performed, and half of the image (angularly) is chosen in a way that excludes any parts of the ROI outside the original area of the mammogram (These edges are generated by superimposing an odd-sized image on a fixed-sized background). The values of the half-grid are normalized in two steps: A stretching of the values over a fixed range increases the contrast within the half-grid, while a normalization of the sum of all the half-grid inputs assures consistency in brightness over all possible half-grids. A statistical thresholding of the ROI is used to determine the approximate "size" of the main feature in the ROI. This size measurement is converted to an equivalent diameter, perturbed by the same scaling perturbation used on the super-pixel grid, then converted into a pair of inputs, using a sine-cosine transformation. These two values, along with the grid super-pixels, are input to the first neural network using a randomly initialized set of neural network weights. The output of the first neural network (restricted between +1 and -1) is compared with the chosen ranking. An error value is calculated and used to correct the neural network weights using a back-propagation learning algorithm known as REM (Recursive Error Minimization). The REM algorithm uses the derivatives of the propagated error to create a more stable convergence than conventional back-propagation. A new set of perturbations is then applied to the next ROI, creating a new half-grid, which is again normalized and applied to the network. This process is repeated until the RMS of the error values drops to a small value, or the change in the error over several iterations becomes very small, whichever occurs first. This signifies that the network is now trained. The number of iterations depends on several conditions, but in general will take about 1000-3000 cycles through the complete set of ROIs.
(26) Once trained, a set of ROIs selected from "unknown" mammograms is processed through the neural network (reference numeral 25 in FIG. 2). No perturbations are done on these inputs, although the grid is rotated through all possible angles, one angular grid space at a time, such that no half-grid includes any "edges". Each half-grid in turn is normalized as above, and applied to the trained network along with the two size inputs from the ROI. The outputs for the rotations are combined, producing two or three statistical outputs for each ROI. These outputs are then used as part of the set of inputs for the second neural network 15, as discussed below.
In response to applicant's argument that Shapiro does not teach luminance gradient or polygon data, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Therefore, the combination of references meets the limitations as presently claimed, and the Examiner most respectfully maintains the rejection.
Conclusion
The prior art made of record but not used in the rejection can be found in the PTO 892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emily C Terrell whose telephone number is (571)270-3717. The examiner can normally be reached Monday - Thursday 7 a.m.-4 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights