DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claims 4, 11 and 18 are hereby objected to for the following reasoning: Claim 4 recites the formula with “contact(…)” in line 3. Claim 4 further states in line 7 that “‘conact’ represents a conact function”. This appears to be a typographical error, as it is believed that both “contact” and “conact” should be “concat” for concatenation. Similar reasoning applies to claim 11 and claim 18.
Specification
The disclosure is objected to because of the following informalities: Paragraph 0051 recites “contact(…)”. Paragraph 0043 recites “conact(…)”. Paragraph 0044 recites “’conact’ represents a conactfunction” in line 3. These appear to be typographical errors, as it is believed that both “contact” and “conact” should be “concat” for concatenation.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 2, 6-9, 13-16 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jia et al. (“Self-supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness Using On-board Videos”).
Regarding claim 1, Jia et al. discloses a depth estimation method, comprising:
dividing an initial image into a plurality of sub-region images, and obtaining a feature vector corresponding to each sub-region image of the plurality of sub-region images by performing a feature extraction on each sub-region image (“rigidly divided the image into many patches, flattening the patches to vectors” at section IVB2, line 2);
inputting the feature vector corresponding to each sub-region image into a depth estimation model, and obtaining depth information corresponding to each feature vector using encoders of the depth estimation model (“To overcome this, we introduce the soft-split multi-layer perceptron (SSMLP) block to promote communication between the patches, thereby simultaneously adjusting the feature map size and reducing the computational and storage costs. Subsequently, the features obtained from the SSMLP are delivered to the Linformer block for extracting the global features. Figure 4 illustrates the detailed structure of the proposed DLBlock” at section IVB2, paragraph 2); and
obtaining a depth image corresponding to the initial image using decoders of the depth estimation model based on the depth information corresponding to each feature vector (“Figure 5 (c) illustrates the structure of the proposed depth decoder, which consists of a few decoder blocks and output heads. For each decoder block illustrated in Fig. 5 (d), a DLBlock is primarily used to perform the feature transformation, following which an upsampling layer is used to increase the resolution” at section IVC, paragraph 2, line 1).
Regarding claim 8, Jia et al. discloses an electronic device, comprising:
at least one processor (implied that the device is a computer); and
a storage device storing a computer program, which when executed by the at least one processor (implied that the computer is programmed to carry out the method), cause the at least one processor to:
divide an initial image into a plurality of sub-region images, and obtaining a feature vector corresponding to each sub-region image of the plurality of sub-region images by performing a feature extraction on each sub-region image (“rigidly divided the image into many patches, flattening the patches to vectors” at section IVB2, line 2);
input the feature vector corresponding to each sub-region image into a depth estimation model, and obtaining depth information corresponding to each feature vector using encoders of the depth estimation model (“To overcome this, we introduce the soft-split multi-layer perceptron (SSMLP) block to promote communication between the patches, thereby simultaneously adjusting the feature map size and reducing the computational and storage costs. Subsequently, the features obtained from the SSMLP are delivered to the Linformer block for extracting the global features. Figure 4 illustrates the detailed structure of the proposed DLBlock” at section IVB2, paragraph 2); and
obtain a depth image corresponding to the initial image using decoders of the depth estimation model based on the depth information corresponding to each feature vector (“Figure 5 (c) illustrates the structure of the proposed depth decoder, which consists of a few decoder blocks and output heads. For each decoder block illustrated in Fig. 5 (d), a DLBlock is primarily used to perform the feature transformation, following which an upsampling layer is used to increase the resolution” at section IVC, paragraph 2, line 1).
Regarding claim 15, Jia et al. discloses a non-transitory storage medium having a computer program stored thereon, which when executed by a processor (implied that the computer is programmed to carry out the method), a depth estimation method is implemented, wherein the depth estimation method comprises:
dividing an initial image into a plurality of sub-region images, and obtaining a feature vector corresponding to each sub-region image of the plurality of sub-region images by performing a feature extraction on each sub-region image (“rigidly divided the image into many patches, flattening the patches to vectors” at section IVB2, line 2);
inputting the feature vector corresponding to each sub-region image into a depth estimation model, and obtaining depth information corresponding to each feature vector using encoders of the depth estimation model (“To overcome this, we introduce the soft-split multi-layer perceptron (SSMLP) block to promote communication between the patches, thereby simultaneously adjusting the feature map size and reducing the computational and storage costs. Subsequently, the features obtained from the SSMLP are delivered to the Linformer block for extracting the global features. Figure 4 illustrates the detailed structure of the proposed DLBlock” at section IVB2, paragraph 2); and
obtaining a depth image corresponding to the initial image using decoders of the depth estimation model based on the depth information corresponding to each feature vector (“Figure 5 (c) illustrates the structure of the proposed depth decoder, which consists of a few decoder blocks and output heads. For each decoder block illustrated in Fig. 5 (d), a DLBlock is primarily used to perform the feature transformation, following which an upsampling layer is used to increase the resolution” at section IVC, paragraph 2, line 1).
Regarding claims 2, 9 and 16, Jia et al. discloses a method, device and medium, wherein the depth estimation model comprises a Transformer model (“1) Linformer: Transformer uses the scaled dot-product attention (SDPA) mechanism to perform feature aggregation, which can be intuitively described as mapping a query and a set of key–value pairs to an output” at section IVB1, line 1), the Transformer model comprises the encoders and the decoders (“Architectures of the proposed depth Linformer network (DLNet), depth decoder, and pose decoder” at Figure 5 description, line 1), and each of the encoders comprises a linear self-attention mechanism and a multilayer perceptron (“Finally, the Linformer block is designed by integrating MHLA and multilayer perceptron (MLP); the detailed structures are shown in Fig. 3” at section IVBA, last sentence; “Subsequently, the feature F2 goes through the Linformer block to capture the global information, thereby obtaining the feature
PNG
media_image1.png
32
218
media_image1.png
Greyscale
. Then, another MLP layer performs the dimension increase to change the feature F3 to
PNG
media_image2.png
30
182
media_image2.png
Greyscale
” at section IVB2, paragraph 6, line 1).
Regarding claims 6, 13 and 20, Jia et al. discloses a method, device and medium wherein the dividing the initial image into the plurality of sub-region images, and obtaining the feature vector corresponding to each sub-region image of the plurality of sub-region images by performing the feature extraction on each sub-region image comprises:
equally dividing the initial image into the plurality of sub-region images according to a length and a width of the initial image (“rigidly divided the image into many patches” at section IVB2, line 2);
extracting features from each sub-region image using a preset feature extraction method, and obtaining a plurality of feature vectors by converting the features extracted from each sub-region image into one feature vector (“rigidly divided the image into many patches, flattening the patches to vectors” at section IVB2, line 2); and
calibrating a position for each of the plurality of feature vectors so that each feature vector comprises position information of the corresponding sub-region image in the initial image (“Let us denote the input feature as Fin 2 RHWdin , where H, W, and din represent the height, width and dimension number of the input feature, respectively. For aggregating the local feature, a moving window of size k x k, stride s, and padding l, is used to reshape the input feature to the sequence” at section IVB2, paragraph 3, line 1).
Regarding claims 7 and 14, Jia et al. discloses a method and device wherein the obtaining the depth image corresponding to the initial image using decoders of the depth estimation model based on the depth information corresponding to each feature vector comprises: generating the depth image based on the depth information corresponding to each feature vector and the corresponding position that is calibrated (“Figure 5 (c) illustrates the structure of the proposed depth decoder, which consists of a few decoder blocks and output heads. For each decoder block illustrated in Fig. 5 (d), a DLBlock is primarily used to perform the feature transformation, following which an upsampling layer is used to increase the resolution” at section IVC, paragraph 2, line 1; each feature is passed through the encoder as demonstrated in Figure 5a and subsequently passed through the decoder in Figure 5C utilizing the same feature parameters F1-F5 to produce the final output).
Allowable Subject Matter
Claims 3-5, 10-12 and 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: while the prior art discloses a linear self-attention mechanism, the prior art does not explicitly disclose that the t-th encoder of the decoders outputs X.sub.t using a formula:
PNG
media_image3.png
36
418
media_image3.png
Greyscale
, wherein, “t” represents an integer greater than 1, “Xt−1” represents an output of the (t−1)-th encoder, “MA” represents the linear self-attention mechanism, and “MLP” represents the multilayer perceptron as required by claims 3, 10 and 17.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Li et al. (NPL Reference) and Yang et al. (NPL Reference) each disclose linear self-attention mechanisms for point cloud processing. Gong et al. (NPL reference) discloses a multi-head attention mechanism in conjunction with MLP modules for image processing. Yao et al. (US 20230095182) and Lee et al. (US 20250356667) disclose depth estimation utilizing transformer based processing of feature vectors. Shi et al. (US 20240171727) discloses a linear self-attention mechanism in conjunction with MLP modules for image processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672
