DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed June 24, 2026, have been fully considered but they are not persuasive.
Claims 26, 28-33, 35-40, and 42-45 are pending in this application and have been considered below. Claims 27, 34, and 41 have been canceled by the applicant.
Applicant argues that Xu is silent regarding the special way how the three feature maps are generated. Xu was never relied on for that; Szegedy supplies the parallel branches “the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector forming the input of the next stage,” the branches being restricted to “filter sizes 1x1, 3x3, and 5x5” (Szegedy p. 3). Xu supplies the padding for size preservation (Xu ¶63). The argument does not attach the combination.
Applicant argues Ex parte Desjardins states the specification must identify an improvement, and the “the claim itself must reflect the disclosed improvement.” The first is met, the second is not. Applicant cites the specification’s ¶23 discloses depth-based filter sizing, “the systems and techniques discussed herein provide a distance based 3D CNN that utilizes multi-scale filters to extract multi-scale features from an input 3D image segment (e.g., a RGB-D image segment). In an embodiment, in the multi-scale filters of the first convolutional layer of the CNN, the size of each cell of each 3D filter has a fixed spatial resolution or size such that the pixel number, during filtering, in each cell differs based on the distance or depth of the volume being filtered from the camera. For each scale or size of filter of the first convolutional layer, there is a pathway to the first fully connected layer of the CNN. The feature vectors provided to the first fully connected layer of the CNN are processed to generate the object recognition data corresponding to the input 3D image segment. Using such techniques, in CNN training, extracted features of objects at large depth/far distance can be used to detect objects at small depth/near distance to camera and vice versa. Furthermore, in implementation, each 3D input image segment (e.g., each RGB-D image segment) is classified into a category as represented by the object recognition data.” The gap is that the independent claims do not recite a 3D filter, cell, depth, distance, nor camera. The conclusion that the claimed invention improves the efficiency of the DNN layers is asserted rather than claimed. Claiming parallel convolutional filters of differing size at the current level of generality applies a mathematical model rather than improving said model.
Therefore, the argued limitations were written broad such that they read upon the cited references or are shown explicitly by the references. As a result, the claims stand as follows.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 26-45 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-21 of U.S. Patent No. 11,880,770. Although the claims at issue are not identical, they are not patentably distinct from each other because the independent claims of the instant application are broader versions of the patented claims. The omitted limitations required by the patented claims – 3D filters, applied to a 3D image segment, that the filters comprise 3D cells of the same spatial size with the first filter comprising more cells than the second, and that the second feature map bypasses the second convolutional layer – are such that the embodiment encompassed by the patented claims would fall within the scope of the instant application’s claims, and the broader claims of the instant application would have been obvious of the narrower patenting claims in view of the common disclosure.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 26, 28-33, 35-40, and 42-45 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) mathematical operations of
applying convolutional filter matrices of different sizes to an input array, generating feature
map arrays, routing feature maps through or bypassing additional layers, and
concatenating feature vectors to generate an output prediction. These operations
constitute mathematical concepts under Alice Corp. v. CLS Bank lnt'l, 573 U.S. 208 (2014),
and Mayo Collaborative Servs. v. Prometheus Labs., Inc., 566 U.S. 66 (2012). This judicial exception is not integrated into a practical application because under the framework of Recentive Analytics, Inc. v. Fox Corp., No. 2022-2027 (Fed. Cir. 2025), defining a novel neural network architecture or training approach that improves ML performance, without more, does not constitute a practical application or technical improvement sufficient to confer eligibility. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements (processor, CRM, convolutional layers, pooling) are well-understood, routine, and conventional components that do not add an inventive concept beyond the abstract idea.
Step 1 -Statutory Category
Claims 26-32: Process
Claims 33-39: Machine (CRM)
Claims 40-45: Machine
All claims satisfy Step 1.
Step 2A, Prong 1 – Judicial Exception
Independent claims 26, 33, 40 all recite:
Providing a 3D input to parallel convolutional layers with filters of different spatial sizes
Generating feature maps from each filter
Routing larger-filter feature maps through additional convolutional layers; smaller filter feature maps bypass those layers
Concatenating resulting feature vectors at a fully connected layer
Generating an object recognition output (prediction)
requiring the first feature map has a different spatial size from the second feature map,
and the third feature map has a same spatial size as the second feature map.
These operations are mathematical in nature – applying weighted filter matrices to input arrays, generating feature map arrays, and concatenating vectors. These are mathematical concepts (mathematical operations/relationships). Under Alice/Mayo Step 2A Prong 1, the claims recite an abstract idea. Under Recentive Analytics, Inc. v. Fox Corp. (Fed. Cir. 2025), training or applying a neural network, even with specific architectural choices, can constitute an abstract mathematical concept when the claims do not recite a specific technological improvement beyond the ML operations themselves.
Step 2A Prong 1: YES, the claims recite abstract idea (mathematical operations/mathematical relationships comprising CNN filter application and feature map generation).
Step 2A, Prong 2 - Practical Application
The claim(s) recite(s) the following additional element(s): a computer processor, a non-transitory computer readable media, convolutional layers, and pooling. These elements are staed generically and merely perform the claimed operations. The independent claims (26, 33, 40) do not recite the depth based filter cell sizing described as the improvement, including any 3D filter, cell, fixed spatial resolution, depth, distance, or camera. The independent claims are written at a high level of architectural generality. The "different spatial size" for filters is a mathematical relationship, not a physical constraint in the independent claims. Under Recentive Analytics, defining a novel CNN architecture to improve ML performance, without more, may still be abstract.
The additional elements therefore, do not integrate the judicial exception into a practical application.
Step 2A Prong 2: NO for claims 26, 33, 40.
Step 2B- Inventive Concept
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements of parallel convolutional branches of differing filter size whose outputs are concatenated is/are shown to be prevalent or in common use in the relevant field by (Szegedy p. 3, “the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector forming the
input of the next stage,” the branches being “restricted to filter sizes 1x1, 3x3, and 5x5.”), which establishes that the element is WURC.
The additional element(s) of padding to preserve spatial size across convolutional layers is/are shown to be widely prevalent or common use in the relevant field by (Xu ¶63, “Padding is typically performed in CNNs to ensure that the image size does not shrink over every convolutional layer”), which establishes the element is WURC.
The additional elements have been considered individually and as an ordered combination. The order combination is no more than the separate elements acting together.
Step 2B: NO, given that multi-scale CNN architectures (e.g., inception modules) were known. Evidence of WURC supports a Step 2B rejection.
Claims
Result
Analysis
26, 33, 40 (independent)
Ineligible
See above
28, 35, 42
Ineligible
Adds "one or more other convolutional layers" –
conventional layer type
29, 36, 42
Ineligible
Adds third filter of different spatial size – mathematical
30, 37, 43
Ineligible
Adds fourth feature map pathway
31, 38, 44
Ineligible
Adds "plurality of cells having the same spatial size"
– begins to recite physical spatial constraint; eligible if physical cell size (¶ [0028]) is added
32, 39, 45
Ineligible
Concatenation step
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 26, 28-33, 35-40, and 42-45 is/are rejected under 35 U.S.C. 103 as being unpatentable over Szegedy et al. (Going Deeper with Convolutions – hereinafter “Szegedy” or “Inception” or “GoogLeNet”) in view of Carreira et al. (US 2020/0125852 A1 – hereinafter “Carreira” or “DeepMind”) further in view of Xu et al. (US 2019/0130573 A1 – hereinafter “Xu”).
Claims 26, 33, and 40.
An apparatus, comprising:
a computer processor for executing computer program instructions; and
one or more non-transitory computer-readable media storing computer program instructions executable by the computer processor to perform operations (Szegedy p. 6 “Although we
used a CPU based implementation only, a rough estimate suggests that the GoogLeNet network could be trained to convergence using few high-end GPUs”) comprising:
providing an input to a first convolutional layer in a neural network, the first convolutional layer having a first filter (Szegedy p. 3 “current incarnations of the Inception architecture are restricted to filter sizes, and 1x1, 3x3, 5x5”; p. 4, Fig. 2(a) shows parallel 1x1, 3x3, and 5x5 convolutions),
generating, in the first convolutional layer, a first feature map from the input and the first filter (Szegedy p. 3 “the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector”; p. 4, Fig. 2(a) shows each parallel convolution branch produces its own output feeding into “Filter concatenation” Szegedy does not use the term “feature map.” The paper uses the terms “output filter banks” and “output vector.” The term “feature map” is standard art-recognized terminology for the output of a convolution layer. The specification ¶21 defines feature maps as outputs of convolution filtering and pooling, which is what Szegedy’s inception module produce.),
providing the input to a second convolutional layer in the neural network, the second convolutional layer having a second filter, wherein the second filter has a different spatial size from the first filter (Szegedy p. 3 “current incarnations of the Inception architecture are restricted to filter sizes, and 1x1, 3x3, 5x5” – the 3x3 and 5x5 branches both receive the same input and have different spatial sizes from each other; p. 4, Fig. 2(a) shows each parallel convolution branch receives the same “Previous layer” input in parallel),
generating, in the second convolutional layer, a second feature map from the input and the second filter (Szegedy same as above – p. 4, Fig. 2(a) shows each parallel convolution branch generates its own output map at a different spatial size),
generating, in one or more other layers arranged after the first convolutional layer in the neural network, a third feature map from the first feature map (Szegedy pp. 3-4 larger filter output pass through subsequent Inception modules – “As these ‘Inception modules’ are stacked on top of each other, their output correlation statistics are bound to vary.” The output of one Inception module feed as the input to the next.; Table 1: inception (3a) output feeds inception (3b), which feeds inception (4a), etc. – multiple layers arranged in sequence each generating feature maps from the prior.), and
generating an output of the neural network based on the third feature map and the second feature map, wherein the output of the neural network comprises a prediction made by the neural network from the input (Szegedy p. 4 “the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector forming the input of the next stage.” Table 1: avg pool → dropout (40%) → linear → softmax with output size 1x1x1000 – this is the prediction output; p. 6 “The ILSVRC 2014 classification challenge involves the task of classifying the image into one of 1000 leaf-node categories”),
(Szegedy p. 3 filter sizes 1x1, 3x3, and 5x5 produce feature maps of different spatial sizes – the 3x3 and 5x5 branches both receive the same input and have different spatial sizes from each other; In Szegedy the “Filter concatenation” where all branches must have the same spatial dimensions. Szegedy achieves this through padding and stride choices, but not through an explicit teaching.).
Szegedy discloses all of the subject matter as described above except for specifically teaching “feature maps.” However, Carreira in the same field of endeavor teaches feature maps (¶37 “feature maps from the upper layers of the 3D convolutional neural networks 110, 120 may be combined and provided to one or more further convolutional neural network layers implemented by combiner 130.”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Szegedy and Carreira before the effective filing date of the claimed invention. The motivation for this combination of references would have been to apply the multi-scale parallel convolution architecture of Szegedy to 3D spatio-temporal data as taught by Carreira, since both reference originate from Google and address convolutional neural network architectures for feature extraction, and Carreira demonstrates that 3D CNNs generate feature maps that can be combined in the same manner as their 2D counterparts.
Szegedy discloses all of the subject matter as described above except for specifically teaching “the third feature map has a same spatial size as the second feature map.” However, Xu in the same field of endeavor teaches the third feature map has a same spatial size as the second feature map (Xu ¶63 “Padding, which is the process of adding a border of "0's" to the activation map before being forwarded to the next convolution layer ( and application of a different filter) can be performed to achieve the same size output image as input image. Padding is typically performed in CNNs to ensure that the image size does not shrink over every convolutional layer.”).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Szegedy and Xu before the effective filing date of the claimed invention. The motivation for this combination of references would have been to apply the well-known padding technique of Xu to the parallel branches of Szegedy Inception module to ensure the feature maps have the same spatial dimensions prior to concatenation, as padding for dimension preservation was a routine practice in convolutional neural networks at the time of filing.
Claims 28 and 35.
The combination of Szegedy, Carreira, and Xu discloses the method of claim 27, wherein the one or more other layers comprise one or more other convolutional layers (Szegedy Table 1: between inception (3a) and the fully connected layers there are inception (3b), (4a), (4b), (4c), (4d), (4e), (5a), (5b) – all convolution layers).
Claims 29, 36, and 42.
The combination of Szegedy, Carreira, and Xu discloses the method of claim 26, further comprising: providing the input to a third convolutional layer in the neural network, the third convolutional layer having a third filter, wherein the third filter has a different spatial size from the first filter and different from the second filter; and generating, in the third convolutional layer, a third feature map from the input and the third filter (Szegedy p. 3 filter sizes 1x1, 3x3, and 5x5 produce feature maps of different spatial sizes).
Claims 30, 37, and 43.
The combination of Szegedy, Carreira, and Xu discloses the apparatus of claim 42, wherein the operations further comprise: generating, in one or more additional layers arranged after the third convolutional layer in the neural network, a fourth feature map from the third feature map, wherein the output of the neural network is generated further based on the fourth feature map (Szegedy Table 1: output of inception (3a) contains three filter-size branches, feeds inception (3b), which in turn generates new feature maps. All of these ultimately contribute to the final softmax output. P. 4, “the next stage can abstract features from the different scales simultaneously” – each subsequent stage generates new feature maps from prior stage outputs, all contributing to the final prediction).
Claims 31, 38, and 44.
The combination of Szegedy, Carreira, and Xu discloses the apparatus of claim 40, wherein the first filter and the second filter comprise a plurality of cells having the same spatial size, and the first filter comprises more cells than the second filter (Szegedy p. 3 filters of size 3x3 and 5x5 can be characterized as having cells (pixels) of the same spatial size, with the 5x5 filter having more cells (25) than the 3x3 filter (9). This is a geometric fact about filter definitions – a 5x5 filter has more cells of the same unit pixel size than a 3x3 filter.)
Claims 32, 39, and 45.
The combination of Szegedy, Carreira, and Xu discloses the apparatus of claim 40, wherein generating the output of the neural network comprises: generating a first feature vector from the third feature map; generating a second feature vector from the second feature map; and concatenating the first feature vector and the second feature map (Szegedy p. 4 “the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector forming the input of the next stage.”; Fig. 2(a) “filter concatenation” block receives all parallel branch outputs and concatenates them; p. 5, Table 1, shows “avg pool → linear” receiving the combined feature representation).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights