DETAILED ACTION
This action is in response to the original filing on October 25, 2023 and the Remarks and Amendments filed on August 21, 2026. Claims 1-8 are pending and have been considered below. Claims 1 and 7 are independent claims. Claims 1-5 and 7 are amended.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In review of Applicant’s amendments, filed August 21, 2026, the objection to the title in the previous office action has been withdrawn.
Claim 7 is no longer being interpreted under 35 U.S.C. 112(f) in view of applicant’s amendment to the claim and remarks filed August 21, 2026.
Response to Arguments
Applicant’s arguments and amendments filed August 21, 2026 regarding the rejections from the previous office action made under 35 U.S.C. 103 have been fully considered but are unpersuasive.
On page 6 of the Remarks, Applicant argues that the amended features of claim 1 “are not disclosed, suggested, or otherwise rendered obvious by over any combination of Yun and Lee.” Applicant cites as reasoning pages 6 and 7 of the previous Office Action, and further asserts on page 7 that “the image in Fig. 1 of Yun, which combines an image of a Saint Bernard and an image of a Poodle, is not an embedded image including a portion representing RBG and a portion representing a distance.” Examiner respectfully disagrees. While Yun fails to explicitly label its images as an “embedded image” or an “RGB image,” Yun does disclose verifying the accuracy of its proposed “CutMix” on the “CIFAR-100” and “ImageNet” databases using “PyramidNet-200” and “ResNet-50,” respectively (Yun, Page 4, Col. 1, ¶1). One of ordinary skill in the art would recognize that the CIFAR-100 and ImageNet databases comprise RGB images and that the PyramidNet-200 and ResNet-50 models are designed to operate on RGB images using three input channels corresponding to Red, Green, and Blue (RGB). Referring back to Figure 1 of Yun, one of ordinary skill in the art can then reasonably infer that the “Original Samples,” or the image of the Saint Bernard and Poodle, are two RGB images which are modified using Yun’s CutMix algorithm (Yun, Page 12, Algorithm A1) to form the “Input Image” or “embedded image” which itself is an RGB image used to train a model, for example “vanilla ResNet-50” (Yun, Page 3, Col. 2, ¶1), which takes RGB images as input (MPEP 2144.01 “ it is proper to take into account not only specific teachings of the reference but also the inferences which one skilled in the art would reasonably be expected to draw therefrom”).
Furthermore, although Yun fails to disclose the embedded image including a “portion representing a distance,” Lee supplies the “distance” portion, or “depth data measured with range sensors such as RGB-D cameras” (Page 2, Col. 2, Section 2.1, ¶1, see claim 1’s rejection below), which functions in substantially the same way as a “distance” as described in the present application’s specification (Page 5, Lines 23-27 “A distance image need not be an image. A distance image may be… a data type different from the data of an RGB image, or may be a matrix replacing distance data obtained by a distance measuring sensor”). All elements of the amended claim are disclosed, suggested, and rendered obvious by the combination of Yun and Lee, as one of ordinary skill in the art may combine Lee’s distance image with the portion of the embedded image of Yun containing the cut-out image, or the Poodle as shown in Figure 1. The motivation to do so is to design a depth estimation method that “outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks” covering “both indoor and outdoor environments” (Lee, Abstract and Page 6, Col. 1, ¶1, see rejections below).
In consideration of this conclusion, the rejections under 35 U.S.C. 103 still stand.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over Yun et al. (“CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features,” 2019, hereinafter Yun) in view of Lee et al. (“From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation” Version 5, 2020, hereinafter Lee).
Regarding claim 1:
Yun teaches a training method (Abstract: “We therefore propose the CutMix augmentation strategy… our CutMix-trained ImageNet classifier”) comprising: obtaining an RGB image and a distance image corresponding to the RGB image (Page 3, Col. 1, Section 3.1, ¶1 “Let x… and y denote a training image and its label, respectively. The goal of CutMix is to generate a new training sample… by combining two training samples (xA, yA) and (xB-, yB)… CutMix replaces an image region with a patch from another training image”).
Yun further teaches cutting a partial area out from the distance image obtained (Page 3, Col. 1, Section 3.1, ¶1 “M… denotes a binary mask indicating where to drop out and fill in from two images… CutMix replaces an image region with a patch from another training image”).
Regarding the limitation generating an embedded image including a portion representing RGB and a portion representing a distance by pasting the partial area cut out from the distance image onto a predetermined area in the RGB image, the predetermined area being located at a position corresponding to a position of the partial area and having a size corresponding to a size of the partial area, Yun teaches generating an embedded image including a portion representing RGB and a portion representing the distance image by pasting the partial area cut out from the distance image onto a predetermined area in the RGB image, the predetermined area being located at a position corresponding to a position of the partial area and having a size corresponding to a size of the partial area (Page 1, Col. 2, Table 1 depicts an image produced by “CutMix,” Page 3, Col. 1, Section 3.1, ¶1-3 “Let x… and y denote a training image and its label, respectively. The goal of CutMix is to generate a new training sample… by combining two training samples (xA, yA) and (xB-, yB)… We define the combining operation as
PNG
media_image1.png
89
491
media_image1.png
Greyscale
Where M… denotes a binary mask indicating where to drop out and fill in from two images… To sample the binary mask M, we first sample the bounding box coordinates B = (rx, ry, rw, rh) indicating the cropping regions on xA and xB. The region B in xA is removed and filled in with the patch cropped from B of xB… we sample rectangular masks M whose aspect ratio is proportional to the original image… With the cropping region, the binary mask… is decided by filling with 0 in the bounding box B, otherwise 1,” Col. 2, Figure 1 depicts “Original Samples” with a “Saint Bernard” or the RGB image and a “Poodle” or the distance image and an input image or embedded image generated by pasting the partial area cut out from the distance image onto a predetermined area or “the cropping region” B in the RGB image, the predetermined area located at a position corresponding to a position of the partial area or “where to drop out and fill in from two images” according to “the binary mask M” and having a size corresponding to a size of the partial area). However, Yun fails to teach a portion representing a distance.
Lee, in the same field of endeavor, teaches a distance (Page 2, Col. 2, Section 2.1, ¶1 “depth data measured with range sensors such as RGB-D cameras or multi-channel laser scanners”).
Regarding the limitation and training a machine learning model, using training data including the embedded image as input data and the distance image as correct answer data, Yun teaches and training a machine learning model, using training data including the embedded image as input data (Page 3, Col. 1, Section 3.1, ¶1 “The goal of CutMix is to generate a new training sample… by combining two training samples… The generated training sample… is used to train the model”). However, Yun fails to teach and the distance image as correct answer data.
Lee teaches and the distance image as correct answer data (Page 2, Col. 2, Section 2.1, ¶1 “In monocular depth estimation, supervised approaches take a single image and use depth data measured with range sensors such as RGB-D cameras or multi-channel laser scanners as ground truth for supervision in training,” Page 8, Col. 2, Section 5, ¶1 “we have presented a supervised monocular depth estimation network,” Page 7, Col. 2, ¶2 “We trained Ours-DenseNet for 50 epochs with 28,654 image-ground truth pairs sampled from the official training and validation set”).
Yun and Lee are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the monocular depth estimation of Lee with the methodology of Yun. The motivation to do so is to design a depth estimation method that “outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks” covering “both indoor and outdoor environments” (Lee, Abstract and Page 6, Col. 1, ¶1).
Regarding claim 2, Yun in view of Lee teaches the training method according to claim 1 (and thus the rejection of claim 1 is incorporated).
Yun further teaches wherein the predetermined area has an area size that is 25% to 75%, inclusive, of the RGB image (Page 3, Col. 1, Section 3.1, ¶1-3 “the combination ratio λ between two data points is sampled from the beta distribution Beta(a, a). In all our experiments, we set a = 1, that is λ sampled from the uniform distribution (0, 1)… we first sample the bounding box coordinates B = (rx, ry, rw, rh) indicating the cropping regions on xA and xB. The region B in xA is removed and filled in with the patch cropped from B of xB… The box coordinates are uniformly sampled according to:
PNG
media_image2.png
85
513
media_image2.png
Greyscale
Making the cropped area ratio rwrh/WH = 1 – λ. With the cropping region, the binary mask M… is decided,” given Equation (2) and “the cropped area ratio,” the area of “the cropping region” can be defined as rwrh = WH(1- λ), hence λ must be between 0.25 and 0.75 for the predetermined area to have an area size that is 25% to 75%, inclusive, of the RGB image, Page 6, Col. 1, Section 4.1.3, ¶3 “’Fixed-size CutMix’ fixes the size of the cropping region (rw, rh) at 16x16 (i.e. λ = 0.75),” Page 8, Fig. 5b depicts various values of “Combination ratio λ” between 0.25 and 0.75, inclusive).
Regarding claim 4, Yun in view of Lee teaches the training method according to claim 1 (and thus the rejection of claim 1 is incorporated).
Lee teaches wherein the machine learning model is trained to learn a relationship between the RGB image and the distance image (Page 2, Col. 2, Section 2.1, ¶1 “depth data measured with range sensors such as RGB-D cameras or multi-channel laser scanners as ground truth,” Page 8, Col. 2, Section 5, ¶1 “a supervised monocular depth estimation network,” Page 7, Col. 1, Section 4.3, ¶1 “KITTI provides the dataset with 61 scenes from “city,” “residential,” “road” and “campus” categories… existing works commonly use a split proposed by Eigen… for the training and test, we also follow it… 697 images covering a total of 29 scenes are used for evaluation and the remaining 32 scenes of 23,488 images are used for training,” Page 7, Col. 2, ¶2 “We… evaluated the proposed method… with a model trained using KITTI’s official split. Apart from the training set, all other settings remain the same as in the experiment using KITTI’s Eigen split. We trained Ours-DenseNet… with 28,654 image-ground truth pairs sampled from the official validation and training set” one of ordinary skill in the art would recognize that training a model with “ground truth pairs sampled from the… training and validation set” encompasses wherein the machine learning model is trained to learn a relationship between the RGB image and the distance image).
Yun and Lee are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the model of Lee with the methodology of Yun. The motivation to do so is to design a depth estimation method that “outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks” covering “both indoor and outdoor environments” (Lee, Abstract and Page 6, Col. 1, ¶1).
Regarding claim 5, Yun in view of Lee teaches the training method according to claim 1 (and thus the rejection of claim 1 is incorporated).
Lee teaches wherein the machine learning model is composed of an encoder network model (Page 8, Col. 1, Section 4.6, ¶1 “the proposed network adopts existing models as an encoder for dense feature extraction”) and an output layer that upsamples, to an output image (Page 4, Col. 1, Section 3.1, ¶1 “at each stage in the decoding phase where internal outputs are recovered to the full resolution with a factor of 2, we employ the proposed local planar guidance (LPG) layer to more effectively relate the features to the desired depth estimation,” Page 3, Fig. 2 depicts the “Local Planar Guidance” layers upsampling inputs with resolutions H/8, H/4, and H/2 back to an output image of resolution H), a low-dimensional feature representation outputted from the encoder network model (Page 4, Col. 1, Section 3.1, ¶1 “the backbone network that we use as a dense feature extractor which produces an H/8 feature map”), the output image having a same size as the RGB image (Page 3, Fig. 2 depicts an “Input” RGB image with resolution H, Page 4, Col. 1, Section 3.1, ¶1 “we follow an encoding-decoding scheme that reduces feature map resolution to H/8 then recovers back to the original resolution H”).
Yun and Lee are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the network architecture of Lee with the methodology of Yun. The motivation to do so is to design a depth estimation method that “outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks” covering “both indoor and outdoor environments” (Lee, Abstract and Page 6, Col. 1, ¶1).
Regarding claim 6, Yun in view of Lee teaches the training method according to claim 1 (and thus the rejection of claim 1 is incorporated).
Lee teaches wherein the machine learning model is composed of an encoder network model and a decoder network model (Page 8, Col. 1, Section 4.6, ¶1 “the proposed network adopts existing models as an encoder,” Page 4, Col. 1, Section 3.1, ¶1 “we follow an encoding-decoding scheme… at each stage in the decoding phase… we employ the proposed local planar guidance (LPG) layer,” Col. 2, Fig. 4 depicts the “local planar guidance layer” or a decoder network model).
Yun and Lee are analogous art to the claimed invention as both are from the same field of endeavor of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the network architecture of Lee with the methodology of Yun. The motivation to do so is to design a depth estimation method that “outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks” covering “both indoor and outdoor environments” (Lee, Abstract and Page 6, Col. 1, ¶1).
Claims 3 and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Yun in view of Lee, and further in view of Yoshikuwa et al. (US 20220292702 A1, hereinafter Yoshikuwa).
Regarding claim 3, Yun in view of Lee teaches the training method according to claim 2 (and thus the rejection of claim 2 is incorporated).
Regarding the limitation wherein the partial area includes an edge portion indicating a contour of an object shown in the RGB image, Yun teaches the partial area (Page 3, Col. 1, Section 3.1, ¶1 “a patch from another training image”) and the RGB image (Page 3, Col. 2, Fig. 1 – “Saint Bernard” in “Original Samples”). However, Yun fails to teach wherein the partial area includes an edge portion indicating a contour of an object shown in the RGB image.
Yoshikuwa, in the same field of endeavor, teaches wherein a partial area includes an edge portion indicating a contour of an object shown in an image (Fig. 6 – 111a-g, 115, ¶108 “the embodiment includes the memory module 115 configured to store the shape of the article A as the object to be captured… the extracting module 111a configured to extract the second image from the first image, the first image being an image of the article A… and the second image being the target range that is a partial area of the first image… the distance detecting module 111b configured to process the second image to detect the distances from at least three parts projected within the target range… the plane estimating module 111c configured to estimate the plane projected within the target range using the distances… the angle detecting module 111d configured to detect the angle of the estimated plane… the contour estimating module 111e configured to estimate, based on the shape of the article A stored in the memory module 115 and the angle of the estimated plane, the contour of the article A projected on the first image, and the controller 100 includes the identifying module 111g configured to identify the article A on the first image based on the contour of the article A,” ¶109 “based on such a contour, the article A can accurately be identified on the first image,” ¶112 “the controller 100 may further include the edge extracting module 111f configured to extract an edge from the image, and the identifying module 111g may identify the article A by comparing a shape of the edge with the contour of the article A”).
Yun and Yoshikuwa are analogous art to the claimed invention as both are in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the image processing modules of Yoshikuwa with the methodology of Yun. The motivation to do so is to reduce image processing and improve image processing speed and accuracy (Yoshikuwa, ¶109 “based on such a contour, the article A can accurately be identified on the first image, which reduces the processing amount required for identifying the article A… the processing amount can be reduced. As a result, the image processing accuracy and speed are improved”).
Regarding claim 7:
Regarding the limitation a training device comprising: a processor; and memory, wherein using the memory, the processor… Yoshikuwa teaches a training device comprising a processor; and memory, wherein using the memory, the processor (Fig. 4 – 100, 101-104, ¶51 “The memory 104 includes a storage device,” ¶52 “The program for the operation of the CPU 101 is stored in the ROM 102 or the memory 104 in advance. The CPU 101 reads the program from the ROM 102 or the memory 104 to the RAM 103, and deploys the program”)…
Claim 7 is a system claim with similar limitations to the method of claim 1. Therefore, claim 7 is rejected under substantially the same rationale as claim 1.
Regarding claim 8, Yun in view of Lee and further in view of Yoshikuwa teaches the training method of claim 1 (and thus the rejection of claim 1 is incorporated).
Yoshikuwa teaches a non-transitory computer-readable recording medium having recorded thereon a computer program for causing a computer to execute a program (Fig. 4 – 100, 101-104, ¶¶51-52 see claim 7 above).
Yun and Yoshikuwa are analogous art to the claimed invention as both are in the same field of endeavor of image processing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the computer-readable medium of Yoshikuwa with the training algorithm of Yun. The motivation to do so is to reduce image processing and improve image processing speed and accuracy (Yoshikuwa, ¶109 “article A can accurately be identified on the first image, which reduces the processing amount required for identifying the article A… the processing amount can be reduced. As a result, the image processing accuracy and speed are improved”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM M LEE whose telephone number is (571)272-4761. The examiner can normally be reached Mon-Fri. 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM M LEE/
Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights