DETAILED ACTION
Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/08/2024 has been considered by the Examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to the mental process of refining a segmentation mask without significantly more.
Regarding claim 1, under step 2A prong 1, the claim recites the mental steps:
-providing a user input including a user-defined area of an image,
-obtaining a second segmentation mask by refining the first segmentation mask based on image data of the image using a computer implemented algorithm
-wherein the algorithm is configured to penalize a violation of the user input.
Under step 2A prong 2, the claim recites additional limitations:
-providing a first segmentation mask associated with an image
This fails to integrate the claim into a practical application because this amounts to well understood routine activity. Similarly, this fails to amount to significantly more than the abstract idea under step 2B.
Regarding claim 2, the claim adds limitations of extending or restricting the first segmentation mask, this amounts to a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 3, the claim adds the limitation of fuzzy processing, this is a mathematical process and fails to remedy the abstract idea of claim 1.
Regarding claim 4, the claim adds penalties, this is a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 5, the claim adds confidence levels, this is a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 6, the claim adds statistical criteria, this is a mathematical process and fails to remedy the abstract idea of claim 1.
Regarding claim 7, the claim adds statistical likelihoods and distributions, these are mathematical processes and fail to remedy the abstract idea of claim 1.
Regarding claim 8, the claim adds an algorithmic function, this is a mathematical process and fails to remedy the abstract idea of claim 1.
Regarding claim 9, the claim adds a comparison of local distributions, this is a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 10, the claim adds a penalization rule, this is a mathematical process and fails to remedy the abstract idea of claim 1.
Regarding claim 11, the claim adds that the segmentation mask is automatically generated by a computer-implemented algorithm, this is well understood routine conventional activity and fails to remedy the abstract idea of claim 1.
Regarding claim 12, the claims adds display devices (generic computing components) and user interaction (mental processes), these fail to remedy the abstract idea of claim 1.
Regarding claim 13, the claim adds iterations, this is a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 14, the claim adds generating a plurality of second segmentation masks and a training data set, this is a mental process and fails to remedy the abstract idea of claim 1.
Regarding claim 15, the claim adds training an AI algorithm, this is well understood, routine, and conventional activity which fails to remedy the abstract idea of claim 1.
Regarding claim 16, the claim adds generic computing components and fails to remedy the abstract idea of claim 1.
Regarding claim 17, the claim adds generic computing components and fails to remedy the abstract idea of claim 1.
Regarding claim 18, the claim adds features, this is data gathering and fails to remedy the abstract idea of claim 1.
Regarding claim 19, the claim adds displaying segmentation masks, this is insignificant extra solution activity and fails to remedy the abstract idea of claim 1.
Regarding claim 20, the claim adds a training data set, this is data gathering and fails to remedy the abstract idea of claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 11-17 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Goyal (US 2022/0270357).
Regarding claim 1, Goyal teaches:
A computer-implemented method of refining a segmentation mask, the method comprising:
providing a first segmentation mask associated with an image, (Goyal [0053] Described herein is the use of a U-net based neural network to predict an initial segmentation mask)
providing a user input comprising including a user-defined area of the image, (Goyal [0057] User guidance input is provided into the system by synthesizing or generating what appear to be captured user mouse events, resulting in inputs which act as a guidance signal) and
obtaining a second segmentation mask by refining the first segmentation mask based on image data of the image using a computer-implemented algorithm, wherein the algorithm is configured to penalize a violation of the user input. (Goyal, see Figure 1B, Refined prediction…See also [0058] The scribbles and the previous interaction are concatenated with the input image to form a 3-channel input for the Interactive CNN. The network is trained iteratively using the simulated user edits to improve segmentation accuracy)
Regarding claim 2, Goyal teaches:
The method of claim 1, wherein at least one of:
the first segmentation mask is associated with a region of interest in the image, (Goyal [0053] Described herein is the use of a U-net based neural network to predict an initial segmentation mask)
the user-defined area of the image is adapted by the user to modify the first segmentation mask, (Goyal [0057] User guidance input is provided into the system by synthesizing or generating what appear to be captured user mouse events, resulting in inputs which act as a guidance signal) and
the user-defined area includes at least one of a first user-defined area adapted by the user to extend the first segmentation mask and a second user-defined area adapted by the user to restrict the first segmentation mask. (Goyal [0057] Specifically (i) “Fore-ground Clicks” c+ are placed within the area of interest, i.e., object to guide the network towards predicting foreground and (ii) “Background Clicks” c_ are plased in the false positive areas which have been incorrectly segmentd as foregoing regions)
Regarding claim 11, Goyal teaches:
The method according to any one of the preceding claims claim 1, wherein the first segmentation mask is automatically generated by a computer-implemented algorithm. (Goyal [0053] Described herein is the use of a U-net based neural network to predict an initial segmentation mask)
Regarding claim 12, Goyal teaches:
The method according to any one of the preceding claims claim 1, wherein at least one of:
one or more of the first and second segmentation mask is displayed on a display device, wherein:
the first segmentation mask is displayed on a display device, (Goyal [0104 processing logic generates an initial prediction image specifying image segmentation by processing the original input images through the base segmentation model to render the initial prediction image))
the user input is received by an input device, (Goyal [0105] processing logic receives user input guidance signals indicating user-guided segmentation refinements to the initial prediction image)
the user input and the first segmentation mask are provided to a computing device, (Goyal [0106] processing logic routes each of (i) the original input images, (ii) the initial predication image, and (iii_ the user input guidance signals to an InterCNN)
the computer-implemented algorithm is performed by the computing device to obtain the second segmentation mask, (Goyal [0107] processing logic generates a refined prediction image specifying refined image segmentation... render the refined segmentation image incorporating the user input guidance signals) and
the second segmentation mask is displayed on the display device. (Goyal [0107] processing logic generates a refined prediction image specifying refined image segmentation... render the refined segmentation image incorporating the user input guidance signals)
Regarding claim 13, Goyal teaches:
The method according to any one of the preceding claims claim 1, wherein: the method is iterated, and wherein the second segmentation mask of an iteration N is used as a first segmentation mask in an iteration N+1. (Goyal [0058] The network is trained iteratively using the simulated user edits to improve segmentation accuracy)
Regarding claim 14, Goyal teaches:
A method of generating a training dataset for an artificial intelligence (AI) algorithm, the method comprising:
applying the method claim 1to a plurality of first segmentation masks to obtain a plurality of second segmentation masks, (Goyal [0109] iteratively repeating each of the following operations …. render new refined prediction image incorporating new user guidance input signals) and
generating the training dataset based on the plurality of second segmentation masks. (Goyal [0109] subsequent to completing the two or more cycles, outputting refined segmentation mask based on application of multiple iterations of the user input guidance signals to the deep learning training framework as the guidance signal)
Regarding claim 15, Goyal teaches:
A method of training an artificial intelligence (AI) algorithm, comprising:
performing the method of t claim 14 to obtain a training dataset, (Please see a full discussion of claim 14 above) and
training the artificial intelligence (AI) algorithm in a supervised manner using the training dataset, wherein the plurality of first segmentation masks is used as input during training and the plurality of second segmentation masks is used as target output. (Goyal [0061] The initial predictions are received from a base segmentation network. These predictions are then compared with ground truth…. network updates the prediction and corresponding scribbles, which are then fed into the model at the next interaction)
Regarding claim 16, Goyal teaches:
A computing device, comprising:
at least one processor, and at least one memory storing computer-executable instructions, the computer-executable instructions when executed by the processor cause the computing device to perform a method according to any one of the preceding claims claim 1. (Goyal [0129-131 processor with memory. Please also see a full discussion of claim 1 above)
Regarding claim 17, Goyal teaches:
The computing device according to claim 16, wherein the computing device is at least one of:
configured to be associated with a display device, such that at least one of the first and second segmentation masks is displayed on the display device, (Goyal [0104 processing logic generates an initial prediction image specifying image segmentation by processing the original input images through the base segmentation model to render the initial prediction image.. See also [0107] processing logic generates a refined prediction image specifying refined image segmentation... render the refined segmentation image incorporating the user input guidance signals) )) and
configured to be associated with an input device configured to receive the user input and to provide the received user input to the computing device. (Goyal [0105] processing logic receives user input guidance signals indicating user-guided segmentation refinements to the initial prediction image)
Regarding claim 19 Goyal teaches:
The method of claim 12, wherein the second segmentation mask is displayed one of adjacent to the first segmentation mask and instead of the first segmentation mask. (Goyal [0107] processing logic generates a refined prediction image specifying refined image segmentation... render the refined segmentation image incorporating the user input guidance signals)
Regarding claim 20, Goyal teaches:
The method of claim 14, wherein the training dataset is based on the plurality of first segmentation masks. (Goyal [0061] The initial predictions are received from a base segmentation network. These predictions are then compared with ground truth…. network updates the prediction and corresponding scribbles, which are then fed into the model at the next interaction)
Allowable Subject Matter
Claims 3-10 and 18 are not rejected under the prior art and would be in condition for allowance if rewritten in independent form and the above rejections under 35 U.S.C. 101 were overcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Molly K Wilburn whose telephone number is (571)272-3589. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Molly Wilburn/Primary Examiner, Art Unit 2666
Prosecution Insights