DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/08/2023 and 08/14/2023 has/have been considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 9 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chen et al (IEEE Internet of Things Journal, 17 November 2020), hereinafter Chen.
-Regarding claim 9, Chen discloses at least one non-transitory machine readable storage medium comprising instructions that, when executed, cause at least one processor (one or more processors and memories have to be used in order to implement Chen’s Fig. 3) to perform operations including comprising (Abstract; Figs. 1-5): implementing a convolution operation in one or more convolutional layers of a convolutional neural network (CNN), including at least (Page 47467, 1st Col., “enrich the input scale of convolutional operation and generate multiscale and multilocation information to improve image classification”, 2nd paragraph, Col., Sec. II-B. Multiscale and Multilocation Contexts in CNN; Figs. 1-3): applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of the convolutional layer (Page 47467, 1st Col., “applying dilated convolution on a part of channels, multiscale contexts may be generated”; Page 7473, 2nd Col., 2nd paragraph; Table V), and applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer (Page 7473, 2nd Col., 2nd and 3rd paragraphs; Page 7479, 2nd Col., Sec. Table V; Figs. 1-3; Table I; Page 7470, 2nd Col., Sec. III-D, 1st paragraph, “kernel”; Page 3472, 1st Col., 2nd paragraph, “three cascaded convolutions with kernel sizes of 1 × 1, 3 × 3 and 1 × 1, respectively”); receiving a set of input data for processing by the CNN; and utilizing the CNN to generate an output (Figs. 2-3), including applying the plurality of dilation rates according to the cyclic pattern in the one or more convolutional layers (Page 7473, 2nd Col., 2nd and 3rd paragraphs; Tables V-VII; Figs. 1-3).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6-8 and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (IEEE Internet of Things Journal, 17 November 2020), hereinafter Chen in view of Ma et al (3rd International Conference on Digital Medicine and Image Processing, 6-9 November 2020), hereinafter Ma.
-Regarding claim 1, Chen discloses an apparatus comprising (Abstract; Figs. 1-5): one or more processors to process data (one or more processors have to be used in order to implement Chen’s Fig. 3), including processing for a convolutional neural network (CNN) (page 7648, 2nd Col., Sec. III. Cyclic CNN; Figs. 1-3); and a memory to store data (at least one memory has to be used in order to implement Chen’s Fig. 3), including data for CNN processing (Figs. 1-3); wherein processing of input data by the CNN includes the one or more processors implementing multi-scale convolution in one or more convolutional layers of the CNN (Page 47467, 1st Col., “enrich the input scale of convolutional operation and generate multiscale and multilocation information to improve image classification”, 2nd paragraph, Col., Sec. II-B. Multiscale and Multilocation Contexts in CNN; Figs. 1-3), implementation of the multi-scale convolution into a convolutional layer of the one or more convolutional layers including at least (Figs. 1-3): applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of the convolutional layer (Page 47467, 1st Col., “applying dilated convolution on a part of channels, multiscale contexts may be generated”; Page 7473, 2nd Col., 2nd paragraph; Table V; Table I; Page 7470, 2nd Col., Sec. III-D, 1st paragraph, “kernel”; Page 3472, 1st Col., 2nd paragraph, “three cascaded convolutions with kernel sizes of 1 × 1, 3 × 3 and 1 × 1, respectively), and applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer (Page 7473, 2nd Col., 2nd and 3rd paragraphs; Tables V-VII; Figs. 1-3).
Chen does not disclose omni-scale convolution.
In the same field of endeavor, Ma teaches a method for image segmentation using omni-scale convolution networks (Ma: Abstract; Figures 1-4). Ma further teaches omni-scale convolution (Ma: Figures 2-3). Ma also teaches a plurality of kernels of a kernel lattice of the convolutional layer (Ma: Figures 2-3).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ma by using omni-scale convolution in order to extend a multi-scale network which leverages different scales to include features at scales that might not be explicitly encoded in the network's structure.
-Regarding claim 15, Chen discloses a system comprising (Abstract; Figs. 1-5): one or more processors to process data (one or more processors have to be used in order to implement Chen’s Fig. 3), including processing for a convolutional neural network (CNN) (page 7648, 2nd Col., Sec. III. Cyclic CNN; Figs. 1-3); and a memory to store data (at least one memory has to be used in order to implement Chen’s Fig. 3), including data for CNN processing (Figs. 1-3); and an multi-scale convolution tool to provide support for objection recognition by the CNN in varying scales of object sizes; wherein application of the omni-convolution tool includes at least (Page 47467, 1st Col., “enrich the input scale of convolutional operation and generate multiscale and multilocation information to improve image classification”, 2nd paragraph, Col., Sec. II-B. Multiscale and Multilocation Contexts in CNN; Figs. 1-3): applying a plurality of dilation rates in a plurality of kernels of a kernel lattice of the convolutional layer (Page 47467, 1st Col., “applying dilated convolution on a part of channels, multiscale contexts may be generated”; Page 7473, 2nd Col., 2nd paragraph; Table V; Table I; Page 7470, 2nd Col., Sec. III-D, 1st paragraph, “kernel”; Page 3472, 1st Col., 2nd paragraph, “three cascaded convolutions with kernel sizes of 1 × 1, 3 × 3 and 1 × 1, respectively), and applying a cyclic pattern for the plurality of dilation rates in the plurality of kernels of the convolutional layer (Page 7473, 2nd Col., 2nd and 3rd paragraphs; Tables V-VII; Figs. 1-3).
Chen does not disclose omni-convolution.
In the same field of endeavor, Ma teaches a method for image segmentation using omni-scale convolution networks (Ma: Abstract; Figures 1-4). Ma further teaches omni-convolution (Ma: Figures 2-3). Ma also teaches a plurality of kernels of a kernel lattice of the convolutional layer (Ma: Figures 2-3).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ma by using omni-scale convolution in order to extend a multi-scale network which leverages different scales to include features at scales that might not be explicitly encoded in the network's structure.
-Regarding claim 6, Chen in view of Ma teaches the apparatus of claim 1.
Chen discloses generating an output based at least in part on the multi-scale convolution in one or more convolutional layers of the CNN. (Fig. 3).
Chen does not disclose omni-scale convolution.
In the same field of endeavor, Ma teaches a method for image segmentation using omni-scale convolution networks (Ma: Abstract; Figures 1-4). Ma further teaches omni-scale convolution, and generating an output based at least in part on the omni-scale convolution in one or more convolutional layers of the CNN. (Ma: Figures 2-3).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ma by using omni-scale convolution in order to extend a multi-scale network which leverages different scales to include features at scales that might not be explicitly encoded in the network's structure.
-Regarding claims 7 and 14, Chen in view of Ma teaches the apparatus of claim 1, Chen discloses non-transitory machine readable storage medium of claim 9.
Chen discloses multi-scale convolution is implemented in multiple convolutional layers of the CNN (Fig. 3).
Chen does not disclose omni-scale convolution.
In the same field of endeavor, Ma teaches a method for image segmentation using omni-scale convolution networks (Ma: Abstract; Figures 1-4). Ma further teaches omni-scale convolution, and omni-scale convolution is implemented in multiple convolutional layers of the CNN (Ma: Figures 2-3).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ma by using omni-scale convolution in order to extend a multi-scale network which leverages different scales to include features at scales that might not be explicitly encoded in the network's structure.
-Regarding claim 8, Chen in view of Ma teaches the apparatus of claim 1.
Chen discloses wherein the omni-scale convolution is incorporated in an existing CNN structure (Fig. 3).
Chen does not disclose omni-scale convolution.
In the same field of endeavor, Ma teaches a method for image segmentation using omni-scale convolution networks (Ma: Abstract; Figures 1-4). Ma further teaches omni-scale convolution, and wherein the omni-scale convolution is incorporated in an existing CNN structure (Ma: Figures 2-3).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ma by using omni-scale convolution in order to extend a multi-scale network which leverages different scales to include features at scales that might not be explicitly encoded in the network's structure.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (IEEE Internet of Things Journal, 17 November 2020), hereinafter Chen in view of Ma et al (3rd International Conference on Digital Medicine and Image Processing, 6-9 November 2020), hereinafter Ma, and further in view of Gu et al (2017 4th IAPR Asian Conference on Pattern Recognition), hereinafter Gu.
-Regarding claim 5, Chen in view of Ma teaches the apparatus of claim 1.
Chen in view of Ma does not teach wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution.
However, Gu is an analogous art pertinent to the problem to be solved in this application and teaches a group dilated convolution (GDC) to enlarge receptive filed (Gu: Abstract; Sec. II.). Gu further teaches wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution (Gu: Page 2, 2nd Col., 2nd paragraph, last line – page 3, 1st Col., 1st paragraph, “produces a group convolution (Fig. 2(c)) with four groups if dilation rate becomes 1, which is called Group Dilation Convolution”, 2nd paragraph, “dilation convolution has kernel size of k ×k”).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify the teaching of Chen in view of Ma with the teaching of Gu by using group dilated convolution in order to effectively enlarge the receptive field.
Claim(s) 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (IEEE Internet of Things Journal, 17 November 2020), hereinafter Chen in view of Ben-Arie (Proceedings of the 34th Midwest Symposium on Circuits and Systems, pp. 537-540, Vol. 1, 1991).
-Regarding claim 10, Chen discloses non-transitory machine readable storage medium of claim 9.
Chen does not disclose mixing multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer.
In the same field of endeavor, Ben-Arie teaches a set of neural lattices (Ben-Arie: Abstract; Secs. II-IV; FIGS. 1A-8B). Ben-Arie further teaches mixing multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer (Ben-Arie: Sec. IV., Page 539, 2nd Col., last paragraph, “The CED is simulated by taking the lateral differences of pairs of PES, either horizontally or vertically. Each layer generates another scale”; FIGS. 1A-8B).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ben-Arie by mixing multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer in order to provide a multi-layered lattice structure for efficient image processing with deriving in real time, all the scales in parallel.
-Regarding claim 11, Chen discloses non-transitory machine readable storage medium of claim 9.
Chen does not disclose wherein mixing multi-scale information in two orthogonal dimensions includes the dilation rates of the plurality of kernels alternating along both an input channel and an output channel.
In the same field of endeavor, Ben-Arie teaches a set of neural lattices (Ben-Arie: Abstract; Secs. II-IV; FIGS. 1A-8B). Ben-Arie further teaches wherein mixing multi-scale information in two orthogonal dimensions includes the dilation rates of the plurality of kernels alternating along both an input channel and an output channel (Ben-Arie: FIGS. 1A-8B).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of Chen with the teaching of Ben-Arie by mixing multi-scale information in two orthogonal dimensions of the kernel lattice for the convolutional layer in order to provide a multi-layered lattice structure for efficient image processing with deriving in real time, all the scales in parallel.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (IEEE Internet of Things Journal, 17 November 2020), hereinafter Chen in view of Ben-Arie (Proceedings of the 34th Midwest Symposium on Circuits and Systems, pp. 537-540, Vol. 1, 1991), and further in view of Gu et al (2017 4th IAPR Asian Conference on Pattern Recognition), hereinafter Gu.
-Regarding claim 13, Chen in view of Ben-Arie teaches the apparatus of claim 1.
Chen in view of Ben-Arie does not teach wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution.
However, Gu is an analogous art pertinent to the problem to be solved in this application and teaches a group dilated convolution (GDC) to enlarge receptive filed (Gu: Abstract; Sec. II.). Gu further teaches wherein applying the plurality of dilation rates in the plurality of kernels includes implementing the dilation rates in group convolution (Gu: Page 2, 2nd Col., 2nd paragraph, last line – page 3, 1st Col., 1st paragraph, “produces a group convolution (Fig. 2(c)) with four groups if dilation rate becomes 1, which is called Group Dilation Convolution”, 2nd paragraph, “dilation convolution has kernel size of k ×k”).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify the teaching of Chen in view of Ma with the teaching of Gu by using group dilated convolution in order to effectively enlarge the receptive field.
Allowable Subject Matter
Claims 2-4, 12, and 16-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAO LIU whose telephone number is (571)272-4539. The examiner can normally be reached Monday-Thursday and Alternate Fridays 8:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nay Maung can be reached at (571) 272-7882. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAO LIU/Examiner, Art Unit 2664