DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 17, and therefore claims 18–20 which depend therefrom are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because claim 17 recites “a computer program product … comprising a computer readable storage medium,” but does not recite that the computer readable storage medium is “non-transitory,” and the specification does not provide a clear disavowal of scope or clear definition that would preclude the broadest reasonable interpretation of “computer readable storage medium” from including carrier waves and signals per se.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 6–9, 14–17 and 19–20 are rejected under 35 U.S.C. 103 as being unpatentable over Tanno et al., “Uncertainty Quantification in Deep Learning for Safer Neuroimage Enhancement,” arXiv:1907.13418v1 [eess.IV] 31 Jul 2019, https://doi.org/10.48550/arXiv.1907.13418 (herein “Tanno”) in view of Qian et al., Chinese patent publication no. CN 116109639 B published August 18, 2023 (with reference to provided machine English language translation, herein “Qian”) in view of Chinni, US Patent Application Publication No. US 2020/0210788 A1 (herein “Chinni”).
Regarding claims 1, 9 and 17, with substantive differences between the claims noted in curly brackets {}, deficiencies of Tanno noted in square brackets, and with claim 1 as exemplary, Tanno teaches {a computer-implemented method, comprising: - claim 1 / a monitoring system, comprising: a hardware processor; and a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to: - claim 9 / A computer program product for monitoring an image data stream, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a hardware processor to cause the hardware processor to: - claim 17} (Tanno Abstract and bottom of page 16 –page 17, methods of determining uncertainty in image processing tasks including super resolution performed on a CPU and GPU with memory for GPU parallelization, where CPU/GPU is a processor and the memory is also a computer program product with a computer readable storage medium executing program instructions therein)
generating detection outputs for an image over multiple iterations by applying a dropout randomly to a different convolutional layer of a learning model for each iteration (Tanno figure 5, pages 10–11, section 3.5, a low-res input image is input to convolutional layers where a variational dropout is applied to every convolution layer by applying Gaussian noise (randomly) according to eq(8) which is performed on every forward pass (each iteration));
[clustering, on labels,] the detection outputs for each iteration (Tanno page 11, figure 5, HR prediction (detection) is output for each low-resolution input sub-volume x);
[computing a total surface area for clusters over the iterations];
computing a confidence for the image (Tanno page 12, model-independent uncertainty (confidence) due to data (the image) is calculated as an intrinsic uncertainty according to equation 16) [using the total surface area for the clusters as an uncertainty score]; and
disabling [a {detection – claims 9 and 17 only} system] if the confidence is below [a threshold] (Tanno bottom of page 26–page 27, predictive uncertainty is used to decide when to abstain from predictions in high-risk regions of images, for example, the original input low-resolution image is processed for high-resolution prediction only in locations with sufficiently low uncertainty (thus, disabling high-resolution prediction of regions with high uncertainly (low confidence))).
While Tanno is directed towards processing images using a convolutional network to output determinations of image confidence (intrinsic uncertainty), Tanno does not explicitly teach or suggest clustering on labels, computing a total surface area for clusters over the iterations, using the total surface area for the clusters as an uncertainty score, or that it is specifically a system that is disabled if the confidence level is below a threshold.
However, Qian teaches clustering on labels (Qian page 11, fig. 6, abnormal detection probability graph and semantic segmentation graph are respectively clustered, and a two-value (labels) segmentation graph is obtained therefrom), computing a total surface area for clusters over the iterations, using the total surface area for the clusters as an uncertainty score (Qian page 10, a cluster area is obtained (computing) and clustering area is judged (used) to determine overlapped areas which in turn are used to determine the confidence degree (uncertainty score), where this processing is described in fig. 7, bottom of page 13 regarding the second fusion model 80 shown as being in a loop of processing blocks, thus as part of/over the iterations).
Chinni teaches disabling a {detection} system if the confidence is below a threshold (Chinni ¶¶18–19, a confidence value is calculated for predictions/image processing performed upon the input image, when the difference between the confidence values is less than a predetermined threshold, the electronic processor determines that the image data received from the camera is outside the predetermined range that the CNN is trained to analyze/detect, and because the autonomous functionality software relies on the prediction generated by the CNN, the electronic processor disables the autonomous functionality of the vehicle controlled by the autonomous functionality software).
Therefore, taking the teachings of Tanno and Qian together as a whole, it would have been obvious to a person having ordinary skill in the art (herein “PHOSITA”) before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the clustering and obtaining cluster areas as disclosed by Qian at least because doing so would improve detection efficiency in the image and ensure reliability of the detection result. Qian bottom of page 1 to top of page 2.
Further taking the teachings of Tanno as modified by Qian and Chinni together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the disabling of a system based on poor confidence in input data being processed as disclosed by Chinni at least because doing so would avoid having a system issue an incorrect analysis and adversely control a system reliant on that analysis when the incorrect analysis occurred due to the data being analyzed not being a type the image analysis software is configured to analyze. See Chinni ¶¶2–3.
Regarding claims 6 and 14, Tanno does not explicitly teach, but Chinni teaches wherein the image is collected from a camera (Chinni ¶19 image data is received (thus collected) by camera 115).
Therefore, taking the teachings of Tanno as modified by Qian and Chinni together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the image as coming from a camera as disclosed by Chinni at least because doing so would avoid having a system issue an incorrect analysis and adversely control a system reliant on that analysis when the incorrect analysis occurred due to the data being analyzed not being a type the image analysis software is configured to analyze. See Chinni ¶¶2–3.
Regarding claims 7, 15 and 19, with substantive differences between the claims noted in curly brackets, and with claim 7 as exemplary, Tanno does not explicitly teach but Chinni teaches wherein the {detection – claims 15 and 19} system includes {a detection/segmentation system that – claim 1} provides {a permission – claim 1 / a service – claims 15 and 19} in accordance with content of the image (Chinni ¶¶18–20, if the image content lies outside of the range that the CNN is trained to analyze, then control of the autonomous functionality is given (a permission) based on an alternative sensor, rather than the camera sensor, where accessing and analyzing the alternative sensor data for autonomous control is also a service).
Further taking the teachings of Tanno as modified by Qian and Chinni together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the disabling of a system based on poor confidence in input data being processed as disclosed by Chinni at least because doing so would avoid having a system issue an incorrect analysis and adversely control a system reliant on that analysis when the incorrect analysis occurred due to the data being analyzed not being a type the image analysis software is configured to analyze. See Chinni ¶¶2–3.
Regarding claims 8, 16 and 20, with claim 8 as exemplary and with substantive differences between the claims noted in curly brackets {}, Tanno does not explicitly teach but Chinni teaches further comprising {the monitoring system – claim 16 / the computer program product further causes the hardware processor to – claim 20} monitoring a data stream of images to detect changes in image quality (Chinni ¶15, CNN makes predictions related to detecting and classifying objects in image data from a camera 115 that controls autonomous (ongoing/streaming) functionality to the vehicle by controlling the braking or paring system, or automatic cruise control, therefore a monitoring of the input image for control).
While Chinni does not explicitly teach a stream of images, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention that Chinni’s input images used for autonomous control of a vehicle would have been a series of images over time (as a stream) since the operations that are being controlled are ongoing continuous operations (braking, parking, and automatic cruise control), and moreover, such a modification would have been a mere duplication of parts with expected results, and therefore as set forth in MPEP §2144.04(VI)(B), would have legal precedent for a finding of obviousness. Further taking the teachings of Tanno as modified by Qian and Chinni together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the autonomous control of a vehicle using images as disclosed by Chinni at least because doing so would avoid having a system issue an incorrect analysis and adversely control a system reliant on that analysis when the incorrect analysis occurred due to the data being analyzed not being a type the image analysis software is configured to analyze. See Chinni ¶¶2–3.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Tanno in view of Qian in view of Chinni, as set forth above regarding the independent claims, further in view of Butoi et al., “UniverSeg: Universal Medical Image Segmentation,” arXiv:2304.06131v1 [cs.CV], April 12, 2023, https://doi.org/10.48550/arXiv.2304.06131 (herein “Butoi”).
Regarding claims 5 and 13, with substantive differences between the claims noted in curly brackets {}, Tanno does not explicitly teach where Butoi teaches { wherein generating the detection outputs includes employing a Universal Learning Model – claim 5 / further comprising a Universal Learning Model to generate the detection outputs – claim 13} (Butoi section 4, model for image segmentation is a universal model which is trained, therefore a universal learning model, and which outputs a 2D binary segmentation of images (detection outputs)).
Therefore taking the teachings of Tanno as modified by Qian and Chinni above, and Butoi together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the image processing of Tanno to include the universal model that is trainable as disclosed by Butoi at least because doing so would allow for solving unseen segmentation tasks without additional training. See Butoi Abstract.
Allowable Subject Matter
Claims 2, 10 and 18, and therefore claims 3–4 which depend from claim 2, and claims 11–12 which depend from claim 10, are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and to overcome any other outstanding rejections, for example the 101 rejection against claim 17 from which claim 18 depends. Specifically, claims 2, 10 and 18 all add the additional limitation of computing the confidence for the image includes computing a standard deviation and average over the total surface area for the clusters over the iterations. The closest cited art includes the combination of Tanno, Qian and Chinni as set forth above regarding the independent claims. Qian is relied upon for teaching specifically calculating the total surface area for the clusters, but Qian does not teach calculating a standard deviation and average over the total surface area for the clusters. Such a claim limitation requires multiple total surface areas to be calculated since a standard deviation and average are aggregate statistics. Further, in considering the claim as a whole, the standard deviation and average statistics are calculated over the total surface area for the clusters over the iterations, where the clusters are understood to be the result of the earlier clustering step, performed on the detection outputs for each iteration, the detection outputs in turn resulting from applying a dropout randomly to a different convolutional layer of a learning model processing an image. Therefore, while standard deviation and averaging statistics are known to the PHOSITA, they would not be obviously combined with Tanno, Qian and Chinni for computing over the total surface area for the clusters over the iterations, as specifically claimed. Therefore, claims 2, 10 and 18, and claims depending therefrom patentably distinguish over the cited art of record.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kingma et al., “Variational Dropout and the Local Reparameterization Trick,” Advances in Neural Information Processing Systems 28 (NIPS 2015), directed towards applying dropouts to individual layers of a neural network.
Sangeetha et al., "Detection and Classification of Skin Cancer by Using a Parallel Deconvolutional Network Model," 2022 International Interdisciplinary Humanitarian Conference for Sustainability (IIHC), Bengaluru, India, 2022, pp. 661-670, doi: 10.1109/IIHC55949.2022.10059956, directed towards image segmentation using clustering and determining the surface area of shapes detected in the segmentation.
Radema et al., US Patent Application Publication No. US 2012/0085686 A1, directed towards image processing in a conveyor system that detects anomalies in the images and controls the conveyor system accordingly.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights