CTNF 18/883,195 CTNF 83208 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia 1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority 2. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has received. Oath/Declaration 3. The receipt of Oath/Declaration is acknowledged. Preliminary Amendment 4. The Preliminary Amendment submitted on 12/12/2024 containing amendments to the claims is acknowledged. Information Disclosure Statement 5. The information disclosure statements (IDS) submitted on 11/12/2024, and 11/21/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner. Drawings 6. The drawing(s) filed on 09/12/2024 are accepted by the Examiner. 12-151 AIA 26-51 12-51 Status of Claims 7. Claims 1-20 are pending in this application. Claims 1-4, 7, 14, and 20 were amended in the 12/12/2024 Preliminary Amendment. Double Patenting 08-33 AIA 8. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg , 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman , 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi , 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum , 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel , 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington , 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms . The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA/25, or PTO/AIA/26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp . 08-34 9. Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of US Patent No. 12,124,533 (hereinafter ‘533). Although the claims at issue are not identical, they are not patentably distinct from each other: Regarding Claim 1: Current Application Claim 1: An apparatus comprising: processing circuitry to: receive an input image by a convolution layer of a neural network to generate a plurality of feature maps; perform spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and upsample the spatially sparse feature maps to generate an output image. ‘533 Claim 1: An apparatus comprising: processor circuitry coupled to a memory, the processor circuitry to: receive an input image by a convolution layer of a neural network to generate a plurality of feature maps; perform spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and facilitate upsampling of the spatially sparse feature maps to generate an output image, wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates. Regarding Claim 8: (drawn to a method) Current Application Claim 8: A method comprising: receiving, by one or more processors, an input image by a convolution layer of a neural network to generate a plurality of feature maps; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and upsampling the spatially sparse feature maps to generate an output image. ‘533 Claim 8: A method comprising: receiving, by one or more processors of a computing device, an input image by a convolution layer of a neural network to generate a plurality of feature maps; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and facilitating upsampling of the spatially sparse feature maps to generate an output image, wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates. Regarding Claim 15: (drawn to a computer-readable medium) Current Application Claim 15: At least one computer-readable medium comprising instructions which, when executed, cause a computing device to perform operations comprising: receiving an input image by a convolution layer of a neural network to generate a plurality of feature maps; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and upsampling the spatially sparse feature maps to generate an output image. ‘533 Claim 15: At least one non-transitory computer-readable medium comprising instructions which, when executed, cause a computing device to perform operations comprising: receiving an input image by a convolution layer of a neural network to generate a plurality of feature maps; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps; and facilitating upsampling of the spatially sparse feature maps to generate an output image, wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates. 10. As shown in the tables above, it is clear that all the elements of the application claims [1, 8 and 15] are to be found in patent claims [1, 8 and 15], as the application claims [1, 8 and 15] fully encompasses patent claims [1, 8 and 15]. The difference between the application claims [1, 8 and 15] and the patent claims lies in the fact that the patent claims includes more elements and is thus more specific. Thus the invention of claims [1, 8 and 15] of the patent is in effect a “species” of the “generic” invention of the application claims [1, 8 and 15]. It has been held that the generic invention is “anticipated” by the “species”. See In re Goodman , 29 USPQ2d 2010 (Fed. Cir. 1993). 11. For example, independent claim 1 of the application does not require “ wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates .” that is claimed in the patent claim 1. For example, independent claim 8 of the application does not require “ wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates .” that is claimed in the patent claim 8. For example, independent claim 15 of the application does not require “ wherein the plurality of feature maps are used to generate masks having probabilistic masks associated with regions of the plurality of maps, wherein the upsampling of the spatially sparse feature maps is facilitated based on one or more predetermined upsampling rates .” that is claimed in the patent claim 15. 12. Since application claims [1, 8 and 15] are anticipated by claims [1, 8 and 15] of the patent, they are not patentably distinct from claims [1, 8 and 15] of the patent. 13. Claim 2 of the current application is a broader version of claim 2 of US 12,124,533. 14. Claim 3 of the current application corresponds to claim 3 of US 12,124,533. 15. Claim 4 of the current application corresponds to claim 4 of US 12,124,533. 16. Claim 5 of the current application corresponds to claim 5 of US 12,124,533. 17. Claim 6 of the current application corresponds to claim 6 of US 12,124,533. 18. Claim 7 of the current application corresponds to claim 7 of US 12,124,533. 19. Claim 9 of the current application corresponds to claim 9 of US 12,124,533. 20. Claim 10 of the current application corresponds to claim 10 of US 12,124,533. 21. Claim 11 of the current application corresponds to claim 11 of US 12,124,533. 22. Claim 12 of the current application corresponds to claim 12 of US 12,124,533. 23. Claim 13 of the current application corresponds to claim 13 of US 12,124,533. 24. Claim 14 of the current application corresponds to claim 14 of US 12,124,533. 25. Claim 16 of the current application corresponds to claim 16 of US 12,124,533. 26. Claim 17 of the current application corresponds to claim 17 of US 12,124,533. 27. Claim 18 of the current application corresponds to claim 18 of US 12,124,533. 28. Claim 19 of the current application corresponds to claim 19 of US 12,124,533. 29. Claim 20 of the current application corresponds to claim 20 of US 12,124,533. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 30. 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. 07-103 AIA 31. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. Claims 15-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because claims 15-20 covers both statutory and non-statutory embodiments (under the broadest reasonable interpretation of the claim when read in light of the specification and in view of one skilled in the art) and embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. “[a] transitory, propagating signal … is not a “process, machine, manufacture, or composition of matter.” Those four categories define the explicit scope and reach of subject matter patentable under 35 U.S.C. § 101; thus, such a signal cannot be patentable subject matter.” ( In re Petrus A.C.M. Nuijten; Fed Cir, 2006-1371, 9/20/2007). Specifically, Applicant’s specification at ¶ [00402] describes “ In some embodiments , a non-transitory computer-readable storage medium has stored thereon data representing sequences of instructions that, when executed by a processor, cause the processor to perform certain operations .” and as a result is drawn to a recording medium that covers both transitory and non-transitory embodiments. Thus, the claims are not eligible subject matter. It is recommended to amend and narrow the claims to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claims. Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 32. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 3 3 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-103 AIA 34. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. 07-15 AIA 35. Claim s 1-2, 7-9, and 14-16 are rejected under 35 U.S.C. 102( a)(1 ) as being anticipated by Huang et al. "HMS-Net: Hierarchical Multi-Scale Sparsity-Invariant Network for Sparse Depth Completion", hereinafter ‘Huang’ . Regarding Claim 1: Huang discloses an apparatus (Huang: Fig 1b ‘Hierarchical Multi-scale encoder-decoder Network (HMS-Net)’) comprising: processing circuitry (Huang: The device, processor, memory and/or program code which enables the disclosed encoding, decoding and convolution operations for image processing in a neural network for computer vision and robotics) to: receive an input image by a convolution layer of a neural network to generate a plurality of feature maps (Huang: Fig. 3: The output of conventional convolution is modified to handle sparse feature map(s) ‘input x’) ; perform spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps (Huang: Page 3432, right column: III Method/A. Sparsity-Invariant Convolution to handle sparse input feature maps; ‘The convolution generates outputs features z(u, v) for each location (u, v). At each spatial location (u, v), the binary sparsity mask mx (u, v) records whether there are input features at this location, i.e., 1 for existing features and 0 otherwise.) ; and upsample the spatially sparse feature maps to generate an output image (Huang: Fig. 3: Illustration of the proposed sparsity-invariant upsampling operation. Described at page 3433, left column; 1) Sparsity-Invariant Bilinear Upsampling) . Regarding Claim 2: Huang further discloses the apparatus of claim 1, wherein the processing circuitry is further to: generate masks based on the plurality of feature maps (Huang: ‘Note that the sparsity mask should always indicate the validity or sparsity of each location of the feature maps. Since the convolution layers in a neural network are generally stacked for multiple times, the output sparsity mask m(sub)z at each stage should be modified to match the output of each stage z. For each output feature location (u, v), if there is at least one valid input location in its receptive field of the previous input, its sparsity mask m(sub)z(u, v) should be updated to 1.) ; and process the plurality of feature maps with the masks to generate the spatially sparse feature maps (Huang: ‘To integrate RGB features into our multi-scale encoder/decoder network, we added an RGB feature path to our proposed network. The network structure is illustrated in Fig. 7(b). The input image is first processed by an RGB sub-network to obtain mid-level RGB features. The structure of the subnetwork follows the first six blocks of the ERFNet [40]. It consists of two downsampling blocks and four residual blocks. The downsampling block has a 2×2 convolution layer with stride 2 and a 2×2 max-pooling layer. The input features are input into the two layers simultaneously, and their results are concatenated along the channel dimension to obtain the 1/2 size feature maps .) . Regarding Claim 7: Huang further discloses the apparatus of claim 1, wherein processing circuitry is coupled to a memory, the processing circuitry comprising one or more of graphics processing circuitry or application processing (Huang: The device, processor, memory and/or program code which enables the disclosed encoding, decoding and convolution operations for image processing in a neural network for computer vision and robotics) . Regarding Claim 8: Huang discloses a method comprising: receiving, by one or more processors (Huang: The device, processor, memory and/or program code which enables the disclosed encoding, decoding and convolution operations for image processing in a neural network for computer vision and robotics) , an input image by a convolution layer of a neural network to generate a plurality of feature maps (Huang: Fig. 3: The output of conventional convolution is modified to handle sparse feature map(s) ‘input x’) ; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps (Huang: Page 3432, right column: III Method/A. Sparsity-Invariant Convolution to handle sparse input feature maps; ‘The convolution generates outputs features z(u, v) for each location (u, v). At each spatial location (u, v), the binary sparsity mask mx (u, v) records whether there are input features at this location, i.e., 1 for existing features and 0 otherwise.) ; and upsampling the spatially sparse feature maps to generate an output image (Huang: Fig. 3: Illustration of the proposed sparsity-invariant upsampling operation. Described at page 3433, left column; 1) Sparsity-Invariant Bilinear Upsampling) . Regarding Claim 9: Huang further discloses the method of claim 8, wherein the performing of spatially sparse convolution comprises: generating masks based on the plurality of feature maps (Huang: ‘Note that the sparsity mask should always indicate the validity or sparsity of each location of the feature maps. Since the convolution layers in a neural network are generally stacked for multiple times, the output sparsity mask m(sub)z at each stage should be modified to match the output of each stage z. For each output feature location (u, v), if there is at least one valid input location in its receptive field of the previous input, its sparsity mask m(sub)z(u, v) should be updated to 1.) ; and processing the plurality of feature maps, by a residual module of the neural network, with the masks to generate the spatially sparse feature maps (Huang: ‘To integrate RGB features into our multi-scale encoder/decoder network, we added an RGB feature path to our proposed network. The network structure is illustrated in Fig. 7(b). The input image is first processed by an RGB sub-network to obtain mid-level RGB features. The structure of the subnetwork follows the first six blocks of the ERFNet [40]. It consists of two downsampling blocks and four residual blocks. The downsampling block has a 2×2 convolution layer with stride 2 and a 2×2 max-pooling layer. The input features are input into the two layers simultaneously, and their results are concatenated along the channel dimension to obtain the 1/2 size feature maps. The main path of the residual block has two sets of 1×3 conv BN ReLU 3×1 conv BN ReLU.) . Regarding Claim 14: Huang further discloses the method of claim 8, wherein the one or more processors are coupled to a memory, the one or more processors comprising one or more graphics processors or one or more application processors (Huang: The device, processor, memory and/or program code which enables the disclosed encoding, decoding and convolution operations for image processing in a neural network for computer vision and robotics) . Regarding Claim 15: Huang discloses at least one computer-readable medium comprising instructions which, when executed, cause a computing device to perform operations (Huang: The device, processor, memory and/or program code which enables the disclosed encoding, decoding and convolution operations for image processing in a neural network for computer vision and robotics) comprising: receiving an input image by a convolution layer of a neural network to generate a plurality of feature maps (Huang: Fig. 3: The output of conventional convolution is modified to handle sparse feature map(s) ‘input x’) ; performing spatially sparse convolution on the plurality of feature maps to generate spatially sparse feature maps (Huang: Page 3432, right column: III Method/A. Sparsity-Invariant Convolution to handle sparse input feature maps; ‘The convolution generates outputs features z(u, v) for each location (u, v). At each spatial location (u, v), the binary sparsity mask mx (u, v) records whether there are input features at this location, i.e., 1 for existing features and 0 otherwise.) ; and upsampling the spatially sparse feature maps to generate an output image (Huang: Fig. 3: Illustration of the proposed sparsity-invariant upsampling operation. Described at page 3433, left column; 1) Sparsity-Invariant Bilinear Upsampling) . Regarding Claim 16: Huang further discloses the computer-readable medium of claim 15, wherein the performing of spatially sparse convolution comprises: generating masks based on the plurality of feature maps (Huang: ‘Note that the sparsity mask should always indicate the validity or sparsity of each location of the feature maps. Since the convolution layers in a neural network are generally stacked for multiple times, the output sparsity mask m(sub)z at each stage should be modified to match the output of each stage z. For each output feature location (u, v), if there is at least one valid input location in its receptive field of the previous input, its sparsity mask m(sub)z(u, v) should be updated to 1.) ; and processing the plurality of feature maps, by a residual module of the neural network, with the masks to generate the spatially sparse feature maps (Huang: ‘To integrate RGB features into our multi-scale encoder/decoder network, we added an RGB feature path to our proposed network. The network structure is illustrated in Fig. 7(b). The input image is first processed by an RGB sub-network to obtain mid-level RGB features. The structure of the subnetwork follows the first six blocks of the ERFNet [40]. It consists of two downsampling blocks and four residual blocks. The downsampling block has a 2×2 convolution layer with stride 2 and a 2×2 max-pooling layer. The input features are input into the two layers simultaneously, and their results are concatenated along the channel dimension to obtain the 1/2 size feature maps. The main path of the residual block has two sets of 1×3 conv BN ReLU 3×1 conv BN ReLU.) . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 3 6 . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA 3 7 . The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-103 AIA 38. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action. 07-23-aia AIA 39. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA 40. Claim s 3-5, 10-12 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. "HMS-Net: Hierarchical Multi-Scale Sparsity-Invariant Network for Sparse Depth Completion" in view of Price et al. (US 11,676,279), hereinafter ‘Price’ . Regarding Claim 3: Huang discloses the apparatus of claim 2, but does not expressly disclose, wherein the processing circuitry is further to: generate a probabilistic mask by performing convolution on the plurality of feature maps; and generate a binary mask based on the probabilistic mask. Price discloses wherein the processing circuitry is further to: generate a probabilistic mask by performing convolution on the plurality of feature maps (Price ‘the object segmentation system 110 utilizes one or both of an object segmentation model 304 or interactive user segmentation 306 to generate the initial object segmentation of a digital image (e.g., the digital image 318 shown in Fig. 3B). For example, the initial object segmentation includes one of a binary segmentation, a probability map, or another boundary edge description (although shown as a simple dashed selection) Col. 10, line 67 – Col. 11, line 7) ; and generate a binary mask based on the probabilistic mask (Price: ‘The segmentation neural network 212 can utilize the probability map in concert with the digital image in determining an object segmentation. For example, at an act 326 the object segmentation system 110 converts the probability map 324 to a binary segmentation mask (e.g., the binary segmentation mask 322) utilizing a graph cut algorithm.’ Col. 15, lines 4-9) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose generating a probabilistic mask by performing convolution on the plurality of feature maps; and generating a binary mask based on the probabilistic mask. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 3. Regarding Claim 4: The proposed combination of Huang in view of Price further discloses the apparatus of claim 3, wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps (Price: ‘Similarly, take for instance a pixel from the probability map 324 indicating a lower likelihood (e.g., 15%) that the pixel is part of the target object in the foreground. In this example, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel a lower cost value to label that pixel as corresponding to the background outside of the target object. Further, by applying the graph cut algorithm, the lower cost value for that pixel falls below (e.g., satisfies) the threshold cost to definitively label that pixel as a background pixel. Therefore, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel as corresponding to a portion of the digital image 318 outside of the target object (e.g., as background).’ Col. 15, lines 44-56) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 4. Regarding Claim 5: The proposed combination of Huang in view of Price further discloses the apparatus of claim 3, wherein the generating of the binary mask comprises generating the binary mask by gating the probabilistic mask with a gating threshold, wherein the probabilistic mask or the binary mask is obtained from neural network training (Price: ‘For example, in some embodiments, the segmentation neural network 212 generates the probability map 324 as comprising other visual indicators that represent a statistical probability on a per-pixel basis (e.g., colored heat indications, added dimensions (e.g., topography), etc.). As another example, the segmentation neural network 212 generates the probability map 324 in non-visual forms.’ Col. 14, line 59 – Col. 14, line 67) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to visually indicate the probabilities that the pixels correspond (or do not correspond) to the target object as disclosed by Price at Col. 14, lines 56-58. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 5. Regarding Claim 10: Huang discloses the method of claim 9, but does not expressly disclose wherein the generating of the masks comprises: Price discloses generating a probabilistic mask by performing convolution on the plurality of feature maps (Price ‘the object segmentation system 110 utilizes one or both of an object segmentation model 304 or interactive user segmentation 306 to generate the initial object segmentation of a digital image (e.g., the digital image 318 shown in Fig. 3B). For example, the initial object segmentation includes one of a binary segmentation, a probability map, or another boundary edge description (although shown as a simple dashed selection) Col. 10, line 67 – Col. 11, line 7) ; and generating a binary mask based on the probabilistic mask (Price: ‘The segmentation neural network 212 can utilize the probability map in concert with the digital image in determining an object segmentation. For example, at an act 326 the object segmentation system 110 converts the probability map 324 to a binary segmentation mask (e.g., the binary segmentation mask 322) utilizing a graph cut algorithm.’ Col. 15, lines 4-9) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose generating a probabilistic mask by performing convolution on the plurality of feature maps; and generating a binary mask based on the probabilistic mask. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 10. Regarding Claim 11: The proposed combination of Huang in view of Price further discloses the method of claim 10, wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps (Price: ‘Similarly, take for instance a pixel from the probability map 324 indicating a lower likelihood (e.g., 15%) that the pixel is part of the target object in the foreground. In this example, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel a lower cost value to label that pixel as corresponding to the background outside of the target object. Further, by applying the graph cut algorithm, the lower cost value for that pixel falls below (e.g., satisfies) the threshold cost to definitively label that pixel as a background pixel. Therefore, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel as corresponding to a portion of the digital image 318 outside of the target object (e.g., as background).’ Col. 15, lines 44-56) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 11. Regarding Claim 12: The proposed combination of Huang in view of Price further discloses the method of claim 10, wherein the generating of the binary mask comprises generating the binary mask by gating the probabilistic mask with a gating threshold, wherein the probabilistic mask or the binary mask is obtained from neural network training (Price: ‘For example, in some embodiments, the segmentation neural network 212 generates the probability map 324 as comprising other visual indicators that represent a statistical probability on a per-pixel basis (e.g., colored heat indications, added dimensions (e.g., topography), etc.). As another example, the segmentation neural network 212 generates the probability map 324 in non-visual forms.’ Col. 14, line 59 – Col. 14, line 67) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to visually indicate the probabilities that the pixels correspond (or do not correspond) to the target object as disclosed by Price at Col. 14, lines 56-58. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 12. Regarding Claim 17: The proposed combination of Huang in view of Price further discloses the computer-readable medium of claim 16, but does not expressly disclose wherein the generating of the masks comprises: generating a probabilistic mask by performing convolution on the plurality of feature maps; and generating a binary mask based on the probabilistic mask. Price discloses generating a probabilistic mask by performing convolution on the plurality of feature maps (Price ‘the object segmentation system 110 utilizes one or both of an object segmentation model 304 or interactive user segmentation 306 to generate the initial object segmentation of a digital image (e.g., the digital image 318 shown in Fig. 3B). For example, the initial object segmentation includes one of a binary segmentation, a probability map, or another boundary edge description (although shown as a simple dashed selection) Col. 10, line 67 – Col. 11, line 7) ; and generating a binary mask based on the probabilistic mask (Price: ‘The segmentation neural network 212 can utilize the probability map in concert with the digital image in determining an object segmentation. For example, at an act 326 the object segmentation system 110 converts the probability map 324 to a binary segmentation mask (e.g., the binary segmentation mask 322) utilizing a graph cut algorithm.’ Col. 15, lines 4-9) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose generating a probabilistic mask by performing convolution on the plurality of feature maps; and generating a binary mask based on the probabilistic mask. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 17. Regarding Claim 18: The proposed combination of Huang in view of Price further discloses the computer-readable medium of claim 17, wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps (Price: ‘Similarly, take for instance a pixel from the probability map 324 indicating a lower likelihood (e.g., 15%) that the pixel is part of the target object in the foreground. In this example, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel a lower cost value to label that pixel as corresponding to the background outside of the target object. Further, by applying the graph cut algorithm, the lower cost value for that pixel falls below (e.g., satisfies) the threshold cost to definitively label that pixel as a background pixel. Therefore, the object segmentation system 110 utilizes the graph cut algorithm to assign that pixel as corresponding to a portion of the digital image 318 outside of the target object (e.g., as background).’ Col. 15, lines 44-56) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to better identify objects as disclosed by Price in the Summary of invention. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 18 . Regarding Claim 19: The proposed combination of Huang in view of Price further discloses the computer-readable medium of claim 17, wherein the generating of the binary mask comprises generating the binary mask by gating the probabilistic mask with a gating threshold, wherein the probabilistic mask or the binary mask is obtained from neural network training (Price: ‘For example, in some embodiments, the segmentation neural network 212 generates the probability map 324 as comprising other visual indicators that represent a statistical probability on a per-pixel basis (e.g., colored heat indications, added dimensions (e.g., topography), etc.). As another example, the segmentation neural network 212 generates the probability map 324 in non-visual forms.’ Col. 14, line 59 – Col. 14, line 67) . Huang in view of Price are combinable because they are from the same field of endeavor of image processing; e.g. both disclose using neural networks to identify objects and improving the accuracy of masks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the probabilistic mask comprises scores for each pixel in each of the plurality of feature maps, wherein the probabilistic mask comprises scores for each region in each of the plurality of feature maps. The suggestion/motivation for doing so is to visually indicate the probabilities that the pixels correspond (or do not correspond) to the target object as disclosed by Price at Col. 14, lines 56-58. Therefore, it would have been obvious to combine Huang with Price to obtain the invention as specified in claim 19 . Allowable Subject Matter 12-151-08 AIA 07-43 12-51-08 41. Claim s 6, 13 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. 13-03-01 AIA 42. The following is a statement of reasons for the indication of allowable subject matter: Regarding Claim 6: None of the prior art cited disclose or suggest the apparatus of claim 1, wherein the upsampling of the spatially sparse feature maps is performed at an upsampling rate of 2, 3, 4, or 5, wherein the neural network is a deep neural network (DNN). Regarding Claim 13: None of the prior art cited disclose or suggest the method of claim 8, wherein the upsampling of the spatially sparse feature maps is performed at an upsampling rate of 2, 3, 4, or 5, wherein the neural network is a deep neural network (DNN). Regarding Claim 20: None of the prior art cited disclose or suggest the computer-readable medium of claim 15, wherein the upsampling of the spatially sparse feature maps is performed at an upsampling rate of 2, 3, 4, or 5, wherein the neural network is a deep neural network (DNN), wherein the computing device comprises a processor coupled to a memory, the processor having a graphics processor or an application processor . Conclusion 07-96 AIA 43. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Pang (CN 109544553 A) discloses a sampling method of deep learning, deep convolutional neural network, the up-sampling method can be applied to image semantic segmentation (Semantic Segmentation of Image), image super-resolution reconstruction, image de-noising, image processing and video processing applications. separated image semantics of the learning based on depth (mainly is a deep convolutional neural network), image super-resolution reconstruction, image de-noising, image processing and video processing. 44. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL R MCLEAN whose telephone number is (571)270-1679. The examiner can normally be reached Monday-Thursday, 6AM - 4PM, PST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /NEIL R MCLEAN/Primary Examiner, Art Unit 2681 Application/Control Number: 18/883,195 Page 2 Art Unit: 2681 Application/Control Number: 18/883,195 Page 3 Art Unit: 2681 Application/Control Number: 18/883,195 Page 4 Art Unit: 2681 Application/Control Number: 18/883,195 Page 5 Art Unit: 2681 Application/Control Number: 18/883,195 Page 6 Art Unit: 2681 Application/Control Number: 18/883,195 Page 7 Art Unit: 2681 Application/Control Number: 18/883,195 Page 8 Art Unit: 2681 Application/Control Number: 18/883,195 Page 9 Art Unit: 2681 Application/Control Number: 18/883,195 Page 10 Art Unit: 2681 Application/Control Number: 18/883,195 Page 11 Art Unit: 2681 Application/Control Number: 18/883,195 Page 12 Art Unit: 2681 Application/Control Number: 18/883,195 Page 13 Art Unit: 2681 Application/Control Number: 18/883,195 Page 14 Art Unit: 2681 Application/Control Number: 18/883,195 Page 15 Art Unit: 2681 Application/Control Number: 18/883,195 Page 16 Art Unit: 2681 Application/Control Number: 18/883,195 Page 17 Art Unit: 2681 Application/Control Number: 18/883,195 Page 18 Art Unit: 2681 Application/Control Number: 18/883,195 Page 19 Art Unit: 2681 Application/Control Number: 18/883,195 Page 20 Art Unit: 2681 Application/Control Number: 18/883,195 Page 21 Art Unit: 2681 Application/Control Number: 18/883,195 Page 22 Art Unit: 2681 Application/Control Number: 18/883,195 Page 23 Art Unit: 2681 Application/Control Number: 18/883,195 Page 24 Art Unit: 2681 Application/Control Number: 18/883,195 Page 25 Art Unit: 2681 Application/Control Number: 18/883,195 Page 26 Art Unit: 2681 Application/Control Number: 18/883,195 Page 27 Art Unit: 2681 Application/Control Number: 18/883,195 Page 28 Art Unit: 2681
Prosecution Insights