Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This communication is in response to the amendment filed on 12/29/2025.
Claim 1-4, 7-9, 11-14, 17-19, 21-24 and 27-29 are pending. Claim 5, 6, 10, 15, 16, 20, 25 and 26 have been cancelled and Claim 29 has been newly added.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-4, 7-9, 11-14, 17-19, 21-24 and 27-29 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-4, 8, 9, 11-14, 18, 19, 21-24, 27 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Kar et al. (U.S. Publication No. 2022/0335638) (hereafter, "Kar") in view of HOUBEN et al. (U.S. Publication No. 2024/0054669) (hereafter, "HOUBEN").
Regarding claim 1, Kar teaches an apparatus for scaling a depth prediction ([0026] The depth estimation system may provide a solution to scale/shift ambiguity (or generally referred to as affine ambiguity) in a monocular depth neural network; [0027] the depth estimation system may reduce placement latency by predicting the scale of placed object/planar surface depth using the neural network), the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to ([0035] The depth estimation system 100 includes one or more processors 140 ... The depth estimation system 100 can also include one or more memory devices 142; FIG. 1A): determine, using a trained machine learning system, a predicted depth map for an image ([0036] The neural network 118 is configured to generate a depth map 120 based on the image data 104 captured by the sensor system 102; [0025] a depth estimation system that includes a sensor system that obtains image data, and a neural network configured to generate a depth map based on an image frame of the image data (e.g., using a single image frame to generate a depth map)), the predicted depth map including a plurality of predicted depth values ([0037] the neural network 118 is considered a monocular depth neural network because the neural network 118 predicts a depth map 120 based on a single image frame 104a. The neural network 118 is configured to predict pixel-wise depth from the image frame 104a; [0040] a dense depth map (e.g., depth map 120) provides depth values (e.g., non-metric depth values) for a large number of pixels in the image or all of the pixels in the image)), each a predicted depth value corresponding to a pixel of the image ([0036] A depth map 120 may refer to an image where each pixel represents the depth value according to a non-metric scale (e.g., 0 to 1) for the corresponding pixel in the image); obtain a plurality of depth values for the image ([0040] The depth estimation system 100 includes a depth estimate generator 106 that obtains depth estimates 108 (e.g., metric depth estimates) associated with the image data 104. The depth estimates 108 may include depth values in a metric scale for some of the pixels in the image data 104) from a tracker ([0040] a depth estimate generator 106; [0042] Referring to FIG. 1B, the depth estimate generator 106 may include a visual inertial motion tracker 160; [0041] the depth estimate generator 106 also obtains pose data 110) configured to determine the plurality of depth values based on one or more feature points between frames ([0043] The visual inertial motion tracker 160 is configured to generate visual feature points 162 that represent the image data 104. The visual feature points 162 are associated with depth estimates 108; [0046] a single camera may be moved around a scene 125 to capture multiple images), the plurality of depth values including depth values for less than all pixels of the image; and ([0040] The depth estimates 108 obtained by the depth estimate generator 106 may be considered sparse depth estimates (e.g., depth estimates for some of the pixels in the image data but not all of them); [0025] The metric depth estimates obtained by the depth estimate generator may be considered sparse depth estimates (e.g., depth estimates for some of the pixels in the image data but not all of them)) scale the predicted depth map for the image using a first representative value computed based on the plurality of depth values and … the plurality of predicted depth values ([0033] The depth estimation system 100 generates a depth map 138 based on depth estimates 108 (obtained from one or more sources) and a depth map 120 … The depth estimation system 100 is configured to convert the depth map 120 having the first scale to the depth map 138 having the second scale; [0047] the depth estimation system 100 includes a depth map transformer 126 configured to transform the depth map 120 generated by the neural network 118 to a depth map 138 using the depth estimates 108; [0048] The depth map transformer 126 is configured to estimate affine parameters 132 based on the depth map 120 generated by the neural network 118 and the depth estimates 108. The affine parameters 132 include scale 134 and shift 136 of the depth map 120. The scale 134 includes a scale value that indicates the amount of resizing of the depth map 120 ... The depth map transformer 126 is configured to transform the depth map 120 to the depth map 138 using the affine parameters 132 ... the scale 134 and the shift 136 include two numbers (e.g., s=scale, t=shift) which when multiplied and added to the value in each pixel at depth map 120 produce depth map 138 (e.g., D138 (x, y)=s*D120 (x, y)+t), where D120(x, y) is the value in depth map 120 at the pixel location (x, y)) ... The affine parameters 132 can be estimated from a sparse set of depth estimates 108 and then applied to every pixel in the depth map 120).
Kar does not expressly teach … a second representative value computed based on.
However, HOUBEN teaches … a second representative value computed based on ([0110] To correct for this, the predicted height map PDD2 may be scaled using a scaling factor SF1 to the height observed YSh. For example, the scaling factor may be computed as a ratio of an average height of the height observed YSh and an average height of the structure computed from the predicted height map PDD2; [0009] a model (e.g., CNN) configured to predict depth data from an inputted image; obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate; [0104] adjusting of the predicted depth data includes: deriving a predicted average height of the structure from the predicted depth data of the structure, and a real average height of the structure from the observed depth data 708; and scaling the predicted average height to match the real average height).
It would have been obvious before the effective filing date of the claimed invention to one having ordinary skill in the art to modify the method and device of Kar to incorporate the step/system of scaling the predicted height map using an average height of the observed height (depth) data and an average height of the predicted height (depth) map taught by HOUBEN.
The suggestion/motivation for doing so would have been to improve the accuracy of the verification of printed circuit patterns for production by reducing defects ([0006] it is desirable to be able to measure the three dimensional structure of the functional elements to characterize reduce or minimize one or more of defects in the device; [0044] During manufacturing of such circuit patterns, images of the printed circuit patterns are captured to determine whether desired circuit patterns are printed accurately; [0192] the procedures may be distributed across a plurality of processors (e.g., parallel computation) to improve computing efficiency). Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predicted results. Therefore, it would have been obvious to combine Kar and HOUBEN to obtain the invention as specified in claim 1.
Regarding claim 2, the combination of Kar and HOUBEN teaches all the limitations of claim 1 above. Kar teaches wherein the tracker is a six-degree-of-freedom (6DOF) tracker ([0041] the depth estimate generator 106 also obtains pose data 110 ... The pose data 110 may identify a pose (e.g., position and orientation) of a device that executes the depth estimation system 100 (e.g., a smartphone that has the depth estimation system 100). In some examples, the pose data 110 includes a five degree-of-freedom (DoF) position of the device. In some examples, the pose data 110 includes a six DoF position of the device; [0042] Referring to FIG. 1B, the depth estimate generator 106 may include a visual inertial motion tracker 160; [0045] the visual inertial motion tracker 160 is configured to execute a SLAM algorithm).
Regarding claim 3, the combination of Kar and HOUBEN teaches all the limitations of claim 2 above. Kar teaches wherein the 6DOF tracker is configured to use a 6DOF tracking algorithm ([0045] the visual inertial motion tracker 160 is configured to execute a SLAM algorithm which is a tracking algorithm that can estimate the movement of a device (e.g., the smartphone) in space by using the camera 107) to generate the depth values based on matching identified salient feature values across multiple frames and solving for camera motion ([0045] the SLAM algorithm iteratively calculate the position and the orientation (e.g., pose data 110) of the device by analyzing the key points (e.g., visual feature points 162) and descriptors of each image and tracking these descriptors from frame to frame, which can allow for a 3D reconstruction of the environment; [0043] The visual inertial motion tracker 160 is configured to generate visual feature points 162 that represent the image data 104. The visual feature points 162 are associated with depth estimates 108; [0044] The visual feature points 162 are a plurality of points (e.g., interesting points) in 3D space that represent the user's environment ... the user may move her mobile phone's camera around a scene 125 during an AR session 174, where the visual inertial motion tracker 160 may generate visual feature points 162 that represent the scene 125. In some examples, the visual feature points 162 include simultaneous localization and mapping (SLAM) points).
Regarding claim 4, the combination of Kar and HOUBEN teaches all the limitations of claim 1 above. Kar teaches wherein the frames comprise one or more pairs of stereo images ([0046] two slightly different views of the scene are obtained, and these different views are used by the dual-pixel depth estimator 166 to generate the depth estimates 108 … a single camera may be moved around a scene 125 to capture multiple images; [0036] the neural network 118 generates a depth map 120 using two or more image frames 104a).
Regarding claim 8, the combination of Kar and HOUBEN teaches all the limitations of claim 1 above. Kar teaches wherein, to scale the predicted depth map, the at least one processor is configured to ([0033] The depth estimation system 100 generates a depth map 138 based on depth estimates 108 (obtained from one or more sources) and a depth map 120 generated by a neural network 118 … The depth estimation system 100 is configured to convert the depth map 120 having the first scale to the depth map 138 having the second scale): determine a final depth map based on multiplying the predicted depth map with a scale factor ([0048] The scale 134 includes a scale value that indicates the amount of resizing of the depth map 120 ... the scale 134 and the shift 136 include two numbers (e.g., s=scale, t=shift) which when multiplied and added to the value in each pixel at depth map 120 produce depth map 138 (e.g., D138 (x, y) = s*D120 (x, y) + t), where D120(x, y) is the value in depth map 120 at the pixel location (x, y))).
Regarding claim 9, the combination of Kar and HOUBEN teaches all the limitations of claim 8 above. HOUBEN teaches wherein the scale factor includes a relationship between the first representative value and the second representative value ([0104] adjusting of the predicted depth data includes: deriving a predicted average height of the structure from the predicted depth data of the structure, and a real average height of the structure from the observed depth data 708; and scaling the predicted average height to match the real average height. For example, a scaling factor (e.g., a factor SF1 in FIG. 8C) computed as a ratio of an average height computed from the predicted depth data and an average height obtained from the optical scatterometry tool (e.g., Yieldstar)).
With respect to claim 11, arguments analogous to those presented for claim 1, are applicable.
With respect to claim 12, arguments analogous to those presented for claim 2, are applicable.
With respect to claim 13, arguments analogous to those presented for claim 3, are applicable.
With respect to claim 14, arguments analogous to those presented for claim 4, are applicable.
With respect to claim 18, arguments analogous to those presented for claim 8, are applicable.
With respect to claim 19, arguments analogous to those presented for claim 9, are applicable.
With respect to claim 21, arguments analogous to those presented for claim 1, are applicable.
With respect to claim 22, arguments analogous to those presented for claim 2, are applicable.
With respect to claim 23, arguments analogous to those presented for claim 3, are applicable.
With respect to claim 24, arguments analogous to those presented for claim 4, are applicable.
With respect to claim 27, arguments analogous to those presented for claim 8, are applicable.
With respect to claim 28, arguments analogous to those presented for claim 9, are applicable.
Claim 7, 17 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Kar et al. (U.S Publication No. 2022/0335638) (hereafter, "Kar") in view of HOUBEN et al. (U.S. Publication No. 2024/0054669) (hereafter, "HOUBEN") and further in view of ZHENG (U.S Publication No. 2018/0302649).
Regarding claim 7, the combination of Kar and HOUBEN teaches all the limitations of claim 1 above. HOUBEN teaches wherein the first representative value includes a first statistical measure of the plurality of depth values or ([0110] an average height of the height observed YSh; [0009] obtaining a captured image (e.g., SEM image) and observed depth data (e.g., measured height map) of the structure patterned on a substrate; [0104] a real average height of the structure from the observed depth data 708; [0107] FIG. 8B illustrate example of observed depth data obtained from different metrology tools (e.g., AFM, SEM, or Yieldstar)) … and wherein the second representative value includes a first statistical measure of the plurality of predicted depth values of the predicted depth map or ([0110] an average height of the structure computed from the predicted height map PDD2; [0009] a model (e.g., CNN) configured to predict depth data from an inputted image; [0104] deriving a predicted average height of the structure from the predicted depth data of the structure).
HOUBEN does not expressly teach … a second statistical measure value of the plurality of depth values … a second statistical measure of the plurality of predicted depth values of the predicted depth map.
However, ZHENG teaches a second statistical measure value of the plurality of depth values ([0152] S303. Obtain data of the current image block of the depth map; [0163] S304 ... calculate a second average value of the data of the current image block according to the pixel value of the data of the current image block) … a second statistical measure of the plurality of predicted depth values of the predicted depth map (S303 ... obtain prediction data corresponding to the current image block of the depth map, and obtain the predicted pixel value from the prediction data; [0163] S304. Calculate a first average value of the prediction data according to the predicted pixel value).
It would have been obvious before the effective filing date of the claimed invention to one having ordinary skill in the art to modify the method and device of HOUBEN to incorporate the step/system of using the average value of depth data of the depth map and the average value of predicted data for coding a depth map taught by ZHENG.
The suggestion/motivation for doing so would have been to improve the efficiency of coding a depth map by reducing a calculation amount ([0006] methods and apparatuses for coding and decoding a depth map, improving coding and decoding efficiency; [0115] a calculation amount is remarkably reduced when the first average value of the prediction data of the current image block is acquired by means of calculation, improving coding efficiency). Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predicted results. Therefore, it would have been obvious to combine HOUBEN and ZHENG to obtain the invention as specified in claim 7.
With respect to claim 17, arguments analogous to those presented for claim 7, are applicable.
With respect to claim 29, arguments analogous to those presented for claim 7, are applicable.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL C. CHANG whose telephone number is (571)270-1277. The examiner can normally be reached Monday-Thursday and Alternate Fridays 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan S. Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL C CHANG/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669