CTNF 18/717,894 CTNF 97705 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. This communication is in response to the Application No. 18/717,894 filed 06/07/2024. Claims 14-24 and 26-34 are pending. Information Disclosure Statement The information disclosure statement(s) (IDS) submitted on 06/07/2024 has been entered and considered. Initialed copies of the PTO-1449 by the examiner are attached. Drawings 06-22 AIA The drawings are objected to because figures 2 and 4 are illegible to read as the text is degraded . Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance. Specification 07-29 AIA The disclosure is objected to because of the following informalities: At paragraph [50] line 6 of the paragraph should recite, in part, “generate the shared 3D dynamic space convolution kernel) kernel , and a multi-dimensional” in order to avoid typographical errors and/or clarity issues . Appropriate correction is required. Double Patenting 08-33 AIA The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg , 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman , 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi , 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum , 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel , 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington , 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The filing of a terminal disclaimer by itself is not a complete reply to a non-statutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA/25, or PTO/AIA/26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer. Claims 14 and 26 are rejected on the ground of non-statutory double patenting as being unpatentable over claims 21 and 36 of Co-pending Application No. 17/435,657. Although the claims 14 and 26 of this Application No. 18/717,894 and claims at issue are not identical, they are not patentably distinct from each other because the instant application and the conflicting Co-pending application are claiming common subject matter, as follows: This Application No. 18/717,894 Co-pending Application No. 17/435,657 Claim 14: An apparatus for 3-dimensional (3D) dynamic sparse convolution, comprising: interface circuitry; machine readable instructions; and at least one processor circuit to be programmed by the machine readable instructions to: access an input feature map of a 3D data sample; divide the input feature map into a plurality of disjoint input feature map groups; perform a shared 3D dynamic sparse convolution to the plurality of disjoint input feature map groups respectively to obtain a plurality of output feature maps corresponding to the plurality of disjoint input feature map groups, the shared 3D dynamic sparse convolution including a shared 3D dynamic sparse convolutional kernel; and sequentially stack the plurality of output feature maps to obtain an output feature map corresponding to the input feature map. Claim 26: At least one memory comprising instructions to cause at least one processor circuit to: access an input feature map of a 3D data sample; perform input feature map partition to divide the input feature map into a plurality of disjoint input feature map groups; perform a shared 3D dynamic sparse convolution to the plurality of disjoint input feature map groups respectively to obtain a plurality of output feature maps corresponding to the plurality of disjoint input feature map groups, the shared 3D dynamic sparse convolution including a shared 3D dynamic sparse convolutional kernel; and perform output feature map grouping to sequentially stack the plurality of output feature maps to obtain an output feature map corresponding to the input feature map. Claim 21: An apparatus for sparse three-dimensional (3D)) convolution acceleration comprising: one or more processors including a graphics processor to process data; and a memory for storage of data, including feature maps; wherein the one or more processors are to provide for sparse 3D convolution acceleration in a first convolutional layer of a plurality of convolutional layers of a neural network model to facilitate training of the neural network model to perform analysis or processing of image data by applying a shared 3D convolutional kernel/filter to an input feature map (wherein the input feature map is the 3D data sample) derived from the image data to produce an output feature map, including: linearly increasing sparsity of the input feature map by partitioning the input feature map into a plurality (G) of disjoint input groups having dimensions of magnitudes equal to the input feature map, wherein said partitioning includes programmatically selecting active/valid voxels of the input feature map for inclusion in respective input groups of the plurality of disjoint input groups; generation of a plurality of disjoint output groups corresponding to the plurality of disjoint input groups by, for each input group of the plurality of disjoint input groups, performing, a convolution operation using the shared 3D convolutional kernel/filter on all input feature values associated with all or a subset of active/valid voxels of the input group (wherein the 3D shared dynamic sparse convolution is the shared 3D convolutional kernel to subset of active/valid voxels) to produce corresponding output feature values within a corresponding output group of the plurality of disjoint output groups by sliding the shared 3D convolutional kernel/filter according to a stride, left-to-right and top-to-bottom, within a spatial dimension of the input group; and creating the output feature map with enhanced feature representation for use by a subsequent convolutional layer of the plurality of convolutional layers, wherein the output feature map merges all output feature values associated with active/valid voxels of each of the plurality of disjoint output groups into the output feature map by sequentially stacking the plurality of disjoint output groups. Claim 36: A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by one or more processors, including a graphics processor, causes the one or more processors to, as part of training of a neural network model to perform analysis or processing of image data: receive, by a first convolutional layer of a plurality of convolutional layers of the neural network model, an input feature map derived from the image data (wherein the input feature map is the 3D data sample) ; linearly increase sparsity of the input feature map, by partitioning, by the first convolutional layer, the input feature map into a plurality (G) of disjoint input groups having dimensions of magnitudes equal to the input feature map; generate, by the first convolutional layer, a plurality of disjoint output groups corresponding to the plurality of disjoint input groups by, for each input group of the plurality of disjoint input groups, performing, a convolution operation using a shared 3D convolutional kernel/filter associated with the convolutional layer on all input feature values of all or a subset of active/valid voxels of the input group (wherein the 3D shared dynamic sparse convolution is the shared 3D convolutional kernel to subset of active/valid voxels) to produce corresponding output feature values within a corresponding output group of the plurality of disjoint output groups by sliding the shared 3D convolutional kernel/filter according to a stride, left-to-right and top-to-bottom, within a spatial dimension of the input group; and create, by the first convolutional layer, an output feature map with enhanced feature representation for use by a subsequent convolutional layer of the plurality of convolutional layers, wherein the output feature map merges all output feature values associated with active/valid voxels of each of the plurality of disjoint output groups into the output feature map by sequentially stacking the plurality of disjoint output groups Claims 15 and 27 are rejected on the grounds of non-statutory double patenting as being unpatentable over claim 21 of Co-pending Application No. 17/435,657 in view of Qi et al. (“Multi-dimensional sparse models”, 2017). Regarding claim 15, Co-pending Application No. 17/435,657 discloses the apparatus of claim 14, Co-pending Application No. 17/435,657 fails to explicitly teach a multi-dimensional mechanism. However, Qi in the same field of endeavor teaches a multi-dimensional mechanism (“MD [multidimensional] synthesis/analysis sparse models to utilize multilinearity of tensors (MD signals) to depict the diversity of MD signals by adaptive separable dictionaries. We construct the redundant basis of the space of multi linear maps with a sparsity constraint and establish the relationship between the redundant basis and MD dictionaries” Qi, pg. 164 Col 1). Therefore, it would have been obvious to one of ordinary skill in the art to combine Co-pending Application No. 17/435,657 and Qi before the effective filing date of the claimed invention. The motivation for this combination of references would have been to greatly reduce memory usage while still achieving competitive results of multidimensional sparse models (Qi, pg. 164 Col 1). This motivation for the combination of Co-pending Application No. 17/435,657 and Qi is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III). Claim 27 contains identical limitations as claim 15 and is therefore rejected for the same reasons set forth in the rejection of claim 15. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15-03-aia AIA Claim (s) 14, 24, and 26 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Yao et al. (US 20220147791 A1) . Regarding claim 14, Yao discloses An apparatus for 3-dimensional (3D) dynamic sparse convolution (“sparse 3D convolution” Yao, [0041]) , comprising: interface circuitry (“firmware interface 128” Yao, [0050]) ; machine readable instructions; and at least one processor circuit to be programmed by the machine readable instructions to (Yao, [0045]) : access an input feature map of a 3D data sample (“Sparse 3D group convolution 1900 receives as an input an input feature map... each input feature map is 3D (W×H×D)” Yao, [0254]) ; divide the input feature map into a plurality of disjoint input feature map groups (“sparse 3D group convolution 1900 initially partitions the input feature map 1910 into G disjoint input groups 1901a and 1901b” Yao, [0255]; Fig. 19) ; perform a shared 3D dynamic sparse convolution to the plurality of disjoint input feature map groups respectively to obtain a plurality of output feature maps corresponding to the plurality of disjoint input feature map groups, the shared 3D dynamic sparse convolution including a shared 3D dynamic sparse convolutional kernel (“sparse 3D group convolution 1900 encourages full use of sparsity for more efficient 3D sparse convolutions. Specifically, in one embodiment, sparse 3D group convolution 1900 has three interdependent operations: (1) input feature map partitioning; (2) shared sparse 3D convolutions; and (3) output feature map merging” Yao, [0253]; “a shared set of one or more sparse 3D convolutions are applied to each of the G disjoint input groups to produce a corresponding set of G disjoint output groups. The one or more sparse 3D convolutions may represent a set of 3D convolution kernels/filters learned by a neural network model that are stacked within a hierarchical structure” Yao, [0259]) ; and sequentially stack the plurality of output feature maps to obtain an output feature map corresponding to the input feature map (“the output features values of active/valid voxels of the output groups are gathered/merged to generate an output feature map containing all active/valid voxels previously distributed across the G disjoint output groups. In one embodiment, the output groups resulting from the shared sparse 3D convolution(s) are sequentially stacked” Yao, [0275]) . Regarding claim 24 , Yao discloses the apparatus of claim 14, wherein one or more of the at least one processor circuit is to partition the input feature map based on the feature locations (“As a result of the partitioning process, the first set of valid/active feature locations (the black blocks) are partitioned into input group 1901 a and the second set of valid/active feature locations (the gray blocks) are partitioned into input group 1901 b ” Yao, [0256]; Fig. 19) . Regarding claim 26 , Yao discloses at least one memory comprising instructions to cause at least one processor circuit to (Yao, [0045]) : access an input feature map of a 3D data sample (“Sparse 3D group convolution 1900 receives as an input an input feature map... each input feature map is 3D (W×H×D)” Yao, [0254]) ; perform input feature map partition to divide the input feature map into a plurality of disjoint input feature map groups (“sparse 3D group convolution 1900 initially partitions the input feature map 1910 into G disjoint input groups 1901a and 1901b” Yao, [0255]; Fig. 19) ; perform a shared 3D dynamic sparse convolution to the plurality of disjoint input feature map groups respectively to obtain a plurality of output feature maps corresponding to the plurality of disjoint input feature map groups, the shared 3D dynamic sparse convolution including a shared 3D dynamic sparse convolutional kernel (“sparse 3D group convolution 1900 encourages full use of sparsity for more efficient 3D sparse convolutions. Specifically, in one embodiment, sparse 3D group convolution 1900 has three interdependent operations: (1) input feature map partitioning; (2) shared sparse 3D convolutions; and (3) output feature map merging” Yao, [0253]; “a shared set of one or more sparse 3D convolutions are applied to each of the G disjoint input groups to produce a corresponding set of G disjoint output groups. The one or more sparse 3D convolutions may represent a set of 3D convolution kernels/filters learned by a neural network model that are stacked within a hierarchical structure” Yao, [0259]) ; and perform output feature map grouping to sequentially stack the plurality of output feature maps to obtain an output feature map corresponding to the input feature map (“the output features values of active/valid voxels of the output groups are gathered/merged to generate an output feature map containing all active/valid voxels previously distributed across the G disjoint output groups. In one embodiment, the output groups resulting from the shared sparse 3D convolution(s) are sequentially stacked” Yao, [0275]) . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim (s) 15 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Yao et al. (US 20220147791 A1) in view of Qi et al. (“Multi-dimensional sparse models”, 2017) . Regarding claim 15, Yao discloses the apparatus of claim 15, wherein the shared 3D dynamic sparse convolutional kernel (“a shared set of one or more sparse 3D convolutions are applied to each of the G disjoint input groups to produce a corresponding set of G disjoint output groups. The one or more sparse 3D convolutions may represent a set of 3D convolution kernels/filters learned by a neural network model that are stacked within a hierarchical structure” Yao, [0259]). Yao discloses all of the subject matter as described above except for specifically teaching a multi-dimensional mechanism. However, Qi in the same field of endeavor teaches a multi-dimensional mechanism (“MD [multidimensional] synthesis/analysis sparse models to utilize multilinearity of tensors (MD signals) to depict the diversity of MD signals by adaptive separable dictionaries. We construct the redundant basis of the space of multi linear maps with a sparsity constraint and establish the relationship between the redundant basis and MD dictionaries” Qi, pg. 164 Col 1). Therefore, it would have been obvious to one of ordinary skill in the art to combine Yao and Qi before the effective filing date of the claimed invention. The motivation for this combination of references would have been to greatly reduce memory usage while still achieving competitive results of multidimensional sparse models (Qi, pg. 164 Col 1). This motivation for the combination of Yao and Qi is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III). Regarding claim 27 , Yao and Qi disclose the at least one memory of claim 26, wherein the shared 3D dynamic sparse convolutional kernel (“a shared set of one or more sparse 3D convolutions are applied to each of the G disjoint input groups to produce a corresponding set of G disjoint output groups. The one or more sparse 3D convolutions may represent a set of 3D convolution kernels/filters learned by a neural network model that are stacked within a hierarchical structure” Yao, [0259]) is modulated with a multi-dimensional mechanism (“MD [multidimensional] synthesis/analysis sparse models to utilize multilinearity of tensors (MD signals) to depict the diversity of MD signals by adaptive separable dictionaries. We construct the redundant basis of the space of multi linear maps with a sparsity constraint and establish the relationship between the redundant basis and MD dictionaries” Qi, pg. 164 Col 1) . Therefore, combining Yao and Qi would meet the claim limitations for the same reasons as previously discussed in claim 15 . Allowable Subject Matter Claim 16, along with its dependent claims 17-23, as well as claim 28 along with its dependent claims 29-34 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims once the Double Patenting rejection, specification and drawing objections are overcome. Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Laddha et al. (US 20210090328 A1) discloses optimizing processing of sparse data, such as 3D point cloud data sets by generating a locality-aware rulebook based on an input unstructured sparse data set, such as a 3D point cloud data set, the locality-aware rulebook storing spatial neighborhood information for active voxels in the input unstructured sparse data set, computing an average receptive field (ARF) value based on the locality aware rulebook, and determining, from a plurality of tile size and loop order combinations, a tile size and loop order combination for processing the unstructured sparse data based on the computed ARF value. Wang et al. (CN 109993297 A) discloses a load balancing accelerator of sparse convolutional neural network accelerator comprises calculating the array of main controller, a data distribution module, a convolution operation, output result cache module, a linear activation function unit, a pooling unit, an online coding unit and off-chip dynamic memory, providing high efficiency calculation, load balance and high utilization of the input excitation and weight data with high reusability. Zhao et al. (CN 116050468 A) discloses generating a dynamic convolution kernel based on the input feature of the predetermined data, wherein each feature channel of the input feature is respectively corresponding to a dynamic convolution kernel; performing channel information exchange between the characteristic channels of the input characteristic, obtaining characteristic information of each characteristic channel of the exchanged channel information; based on the dynamic convolution kernel corresponding to each feature channel, weighting the feature information of the corresponding feature channel to obtain the output value of each feature channel. QU et al (CN 114266900 A) discloses a monocular 3 D target detection method based on dynamic convolution, providing dynamic convolution layer based on depth map generation, using RGB image data as input, then using dense depth network to generate depth map, generating the generated depth map through the filter to generate network training to generate the convolution kernel of the dynamic convolution layer, and applying it to the dynamic convolution layer of the feature extraction network, making the feature extraction considering the depth information. DAULTANI (WO 2018073975 A1) discloses generating a list of non-zero elements from a learned sparse kernel to be used for a convolution layer of the convolution neural network; when performing convolution on an input feature map, loading only elements of the input feature map which correspond to the non-zero elements of the generated list; and performing convolution arithmetic operations using the loaded elements of the input data map and the non-zero elements of the list, thereby reducing the number of operations necessary to generate an output feature map of the convolution layer. Li et al. (CN113326748 A) discloses a neural network behavior identification method using multidimensional related attention model to automatically extract the multi-dimensional attention on the basis of the feature map, using the extracted time, the obvious area on the space and characteristic channel, performing weighted fusion to the generated feature map and obtaining the predicted value, generating the final action identification result. Inquiries Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMMANUEL SILVA-AVINA whose telephone number is (571)270-0729. The examiner can normally be reached Monday - Friday 11 AM - 8 PM EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /EMMANUEL SILVA-AVINA/Examiner, Art Unit 2673 /CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673 Application/Control Number: 18/717,894 Page 2 Art Unit: 2673 Application/Control Number: 18/717,894 Page 3 Art Unit: 2673 Application/Control Number: 18/717,894 Page 4 Art Unit: 2673 Application/Control Number: 18/717,894 Page 5 Art Unit: 2673 Application/Control Number: 18/717,894 Page 6 Art Unit: 2673 Application/Control Number: 18/717,894 Page 7 Art Unit: 2673 Application/Control Number: 18/717,894 Page 8 Art Unit: 2673 Application/Control Number: 18/717,894 Page 9 Art Unit: 2673 Application/Control Number: 18/717,894 Page 10 Art Unit: 2673 Application/Control Number: 18/717,894 Page 11 Art Unit: 2673 Application/Control Number: 18/717,894 Page 12 Art Unit: 2673 Application/Control Number: 18/717,894 Page 13 Art Unit: 2673 Application/Control Number: 18/717,894 Page 14 Art Unit: 2673 Application/Control Number: 18/717,894 Page 15 Art Unit: 2673 Application/Control Number: 18/717,894 Page 16 Art Unit: 2673 Application/Control Number: 18/717,894 Page 17 Art Unit: 2673
Prosecution Insights