DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The presently filed application claims benefit to earlier filed parent application CN202010739666.4 filed July 28, 2020. The copy of the parent application has been filed with the office and claims 1-8, 10-12, 14-18, and 20 are being examined under this filing date.
Claim Status
This action is in response to the application filed on November 18, 2025. Claims 1-2, 12, and 18 are amended. Claims 13 and 19 are cancelled. Claims 21-23 have been added. Thus, claims 1-8, 10-12, 14-18, and 20-23 are pending for examination in this application.
Response to Amendments
Applicant's remarks and amendments filed November 18, 2025 have been entered.
Response to Arguments
Applicant's arguments filed November 18, 2025, have been fully considered but they are not persuasive.
Argument: On page 9, the applicant alleges, “the Office Action likens the noise in Song to correspond to the claimed offset ("the noise is considered to be an offset"). In the Office Action's proposed modification, Song's offset would be reduced to calculate depth information. Without conceding the merits of the Office Action's characterization and solely to expedite prosecution, Applicant amended claim 1 to specify that determining the offset comprises computing a difference between a position of a matching pixel in the reference image and a position of the target pixel in the target image. The Office Action's proposed modification of the offset (i.e., to reduce the offset) would thus interfere with generating a depth image of the target object based on the offset.”
Response: The examiner respectfully disagrees. The examiner in light of the amendment, does not consider the noise as the offset, but the offset is now considered to be the offset of individual dot(s) as seen in Song, Col 12, Lines, 13-25. Therefore, Song would not interfere with generating a depth image of the target object based on the offset.
Song further teaches, that wherein determining the offset comprises computing a difference between a position of a matching pixel in the reference image and a position of the target pixel in the target image (see Song, Col 12, Lines, 13-25, “The device 102 may divide (816) the captured image into one or more captured section(s). The captured section(s) may be centered around a dot that is to be compared to dots in the reference image(s) to find dot matches between the captured section(s) and the reference image(s). The device 102 may determine (818) an (x, y) offset of individual dot(s) in a particular captured section relative to the center dot of the captured section and scale (820) down and quantize the (x, y) offset(s) of the captured section to generate a captured patch. The device 102 may compare (822) the captured patch to one or more reference patches,” quantize the offsets of individual dots in the captured section and compare the offset of the captured patch to one or more reference patches is considered to be computing a difference between a position of a matching pixel in the reference image and a position of the target pixel in the target image).
Therefore Song teaches “wherein determining the offset comprises computing a difference between a position of a matching pixel in the reference image and a position of the target pixel in the target image.”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-8, 10-12, 14-18, and 20-23 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter.
The limitations of independent claim 1 independent claim 12, and independent claim 20, includes “computing a difference between a position of a matching pixel in the reference image and a position of a target pixel in the target image;” The limitation is interpretated as computing a difference between a position of a pixel in the reference image and the corresponding pixel at the same position in the target image. The claim language does not indicate what a matching pixel is, therefore, it is unclear given the current limitations what “a matching pixel” is based on in the claim limitations. The dependent claims do not alleviate the issues of the independent claim and are also rejected under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 10-12, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Song et al, US 9361698 in view of Shi et al, US 20200286248 in further view of Gao et al, CN 110400341.
Regarding claim 1, Song teaches a depth image generation method
performed by an electronic device, the method comprising (see Song Fig. 1 and Col 2, Lines 56-58, “FIG. 1 illustrates a depth sensor device 102 and corresponding method for determining a three-dimensional shape of an object 150”, the depth sensor device 102 is considered to be the electronic device):
emitting structured light to a reference plane (Song Col 2, Lines 10-12, “the device may project the dots onto a flat surface at a fixed distance to create a reference image of the flat surface with the projected dots.”);
imaging the reference plane onto a plurality of first effective pixels and a plurality of second effective pixels of an image sensor to obtain a reference image (see Song Fig. 1 and Col 3, Lines 60-66, “the device 102 may identify individual dots in the captured image and the reference image(s) and may identify exact coordinates and sizes for the dots. For example, the device 102 may identify first pixels associated with the dots (along with coordinates for brightness centers of the dots) and second pixels not associated with the dots”),
wherein the plurality of first effective pixels is positioned in a center region of the image sensor, and the plurality of second effective pixels is positioned surrounding the center region (see Song, Fig. 7A and Col 9, Lines 59-65, "FIG. 7A illustrates an example of determining coordinates for an identified dot. The portion 700 of a captured image may include a 3×3 selected square section 704 including pixels 704-01 to 704-09. 704-05 is the local maximum c in the middle of the selected square section 704. Surrounding the selected square section 704 is a surrounding area 706," plurality of first effective positions is positioned in a center region of the image sensor is considered to be 704 and plurality of second effective pixels is positioned surrounding the center region is considered to be 706);
emitting the structured light to a target object (see Song Fig. 1 and Col 4, Lines 56-58, “Without moving, the device 102 may project the first, second and third dot patterns on object(s) in a scene and capture images for each of the first, second and third dot patterns”);
imaging the target object onto the plurality of first effective pixel to obtain a target image (see Song Col 4, Lines 59-63, “the device 102 may repeat the method illustrated in FIG. 1 for each of the first, second and third dot patterns to generate first, second and third depth information based on the corresponding one or more reference images.” the method in Fig. 1 is being repeated using the target in the captured images therefore it is implied the plurality of first effective images are included to obtain the target image);
matching a respective one of the plurality of first effective pixels in the target image with a corresponding one of the plurality of second effective pixels in the reference image (see Song, Col 19, Lines 8-11, “Based on the captured regions and corresponding matching reference regions, the device 102 may correlate dots in the captured image to dots in the reference image,” the captured regions are considered to contain first effective pixels in a target image and corresponding matching reference regions are considered contain second effective pixels in a reference image; correlate dots are considered to be matching);
determining an offset between the respective one of the plurality of first effective pixels in the target image and the corresponding matching one of the plurality of second effective pixels in the reference image (see Song Col 15, Lines 16-20, ”By reducing the noise and evenly spreading intensities of pixels associated with the dots in the reference image(s), the device 102 may make it easier to correlate a dot in the captured image to a dot in the reference image(s),” the noise is considered to be an offset and upon reducing the noise the device 102 may calculate depth information by correlating the center dot in the captured patch to the center dot in the reference patch),
wherein determining the offset comprises computing a difference between a position of a matching pixel in the reference image and a position of a target pixel in the target image (see Song, Col 12, Lines, 13-25, “The device 102 may divide (816) the captured image into one or more captured section(s). The captured section(s) may be centered around a dot that is to be compared to dots in the reference image(s) to find dot matches between the captured section(s) and the reference image(s). The device 102 may determine (818) an (x, y) offset of individual dot(s) in a particular captured section relative to the center dot of the captured section and scale (820) down and quantize the (x, y) offset(s) of the captured section to generate a captured patch. The device 102 may compare (822) the captured patch to one or more reference patches,” quantize the offsets of individual dots in the captured section and compare the offset of the captured patch to one or more reference patches is considered to be computing a difference between a position of a matching pixel in the reference image and a position of the target pixel in the target image);
and generating a depth image of the target object based on the offset between the respective one of the plurality of first effective pixels in the target image and the corresponding matching one of the plurality of second effective pixels in the reference image (Col 3, Lines 66-67 – Col 4, Lines 1-3, “As will be described in greater detail below with regard to FIGS. 9A-9C, the device 102 may process the captured image and the reference image(s) to remove noise and/or additional information not related to the dots.,” and Col 16, Lines 48-51, “After performing step 1018 and correlating the center dot in the captured patch to the center dot in the reference patch, the device 102 may calculate depth information using any method known to one of skill in the art.”, the noise is considered to be an offset and upon reducing the noise the device 102 may calculate depth information by correlating the center dot in the captured patch to the center dot in the reference patch. Also see Song, Col 17, Lines 30-39, “Alternatively, in step 1012 the device 102 may calculate correlation scores between a captured patch and reference patches associated with multiple reference images having varying depths. For example, the device 102 may compare the captured patch to several reference patches from multiple reference images, despite some of the reference patches having correlated center dots,” the multiple reference images have varying depths which means each the correlation scores between captured patch and reference patches will generate a depth image; the captured patch is considered to be the target object with a plurality of first effective pixels).
Song does not expressively teach
emitting structured light from a light emitter
wherein the light emitter and the image sensor define a base line
emitting the structured light from the light emitter to a target object
and a connection line d between the light emitter and the image sensor on the base line
and a focal length f of the image sensor
However, Shi in a similar invention in the same field of endeavor teaches
emitting structured light from a light emitter (see Shi, Paragraph [0050], “The projector 110 may be an LCD projector or other stable light source.”)
wherein the light emitter and the image sensor define a base line (see Shi, Paragraph [0014], “The light projector may be at a fixed baseline distance from the image sensor”);
emitting the structured light from the light emitter to a target object (see Shi, Paragraph [0050], “The projector 110 is configured to project a unique structured light pattern on one or more objects 120”);
and a connection line d between the light emitter and the image sensor on the base line (see Shi, Paragraph [0022], “The method may further include estimating, with a depth estimator, a depth according to the equation Z=B×F/(P×D), wherein Z is the depth, B is a baseline distance between the light projector)
and a focal length f of the image sensor (see Shi, Paragraph [0022], “The method may further include estimating, with a depth estimator, a depth according to the equation Z=B×F/(P×D), wherein Z is the depth, B is a baseline distance between the light projector and the image sensor, F is a focal length of the image sensor, P is a pixel pitch, and D is a disparity.”).
The combination of Song and Shi are analogous art because they are both in the same field of endeavor of structured light projections and depth estimation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to emit structure light from an LCD projector or other stable light source; for the light projector to be at a baseline distance from the image sensor; to project a unique structured light pattern on one more objects; to binarize the images in the method of Shi in the method of Song to achieve subpixel resolution, thereby increasing resolution and decreasing depth error, without adding any additional hardware to a conventional structured-light-based system (see Shi Paragraph [0029]).
Song in view of Shi does not expressively teach
and a distance R between the reference plane and the base line
However, Gao in a similar invention in the same field of endeavor teaches
and a distance R between the reference plane and the base line (see Gao, Paragraph [0040], “R is the reference calibration plane d<sub>r</sub> away from the transmitting module,”)
The combination of Song, Shi, and Gao are analogous art because they are all in the same field of endeavor of structured light projections and depth estimation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention where R is the reference calibration plane away from the transmitting module in the method of Gao in the method of Song in view of Shi to improve the performance of depth camera applications, and has a compact structure (see Gao Paragraph [0009]).
Regarding claim 2, Song in view of Shi in further view of Gao further teaches the method according to claim 1,
wherein the plurality of first effective pixels is positioned in a designated region that is an image window region of the image sensor (see Song Col 6, Lines 18-23 and Lines 43-51,”A local maximum c may be an individual pixel having a highest intensity in a local region. The device 102 may determine a size and location of a dot including the local maximum c. In some example embodiments, the device 102 may assume a fixed size for a dot, such as a 2 pixel by 2 pixel square, a 3 pixel by 3 pixel square or any other sized square … the device 102 may determine that a best representation of the dot is a rectangle instead of a square. For example, for a rectangle that is x pixels by y pixels, a corresponding search region may be 2x−1 pixels by 2y−1 pixels and may include x*y potential rectangles that include the local maximum c. In these examples, the local maximum c may be used to bisect a selected rectangle in a vertical direction and a horizontal direction).
The rationale of claim 1 has been applied herein.
Regarding claim 3, Song in view of Shi in further view of Gao further teaches the method according to claim 1,
wherein imaging the reference plane onto the plurality of first effective pixels and the plurality of second effective pixels of the image sensor comprises: imaging the reference plane onto the plurality of first effective pixels and all of the plurality of second effective pixels of the image sensor (see Song Fig. 1 and Col 3, Lines 60-66, “the device 102 may identify individual dots in the captured image and the reference image(s) and may identify exact coordinates and sizes for the dots. For example, the device 102 may identify first pixels associated with the dots (along with coordinates for brightness centers of the dots) and second pixels not associated with the dots”).
The rationale of claim 1 has been applied herein.
Regarding claim 4, Song in view of Shi in further view of Gao further teaches the method according to claim 1,
wherein imaging the reference plane onto the plurality of first effective pixels and the plurality of second effective pixels of the image sensor comprises: imaging the reference plane onto the plurality of first effective pixels and different quantities of second effective pixels in the plurality of second effective pixels of the image sensor to obtain at least two reference images with different sizes (see Song Col 4, Lines 47-49, “If different dot patterns are used, different reference images for the respective dot patterns may also be created and used as described below” and Col 15, Lines 26-27, “In addition, the device 102 may repeat these steps for multiple reference images”, in claim 1, it has been established that the dots may be different sizes therefore the quantity of second regions may be different based on the size of the dots used therefore generating difference sized reference images);
and generating the depth image of the target object based on the target image and the reference image comprises: selecting a reference image from the at least two reference images with different sizes, and generating the depth image of the target object according to the target image and the selected reference image (see Song Col 4, Lines 59-63, “the device 102 may repeat the method illustrated in FIG. 1 for each of the first, second and third dot patterns to generate first, second and third depth information based on the corresponding one or more reference images).
The rationale of claim 1 has been applied herein.
Regarding claim 5, Song in view of Shi in further view of Gao further teaches the method according to claim 4,
wherein selecting the reference image from the at least two reference images with different sizes comprises: obtaining a distance from the target object to the reference plane (see Song Col 16, Lines 48-58, “After performing step 1018 and correlating the center dot in the captured patch to the center dot in the reference patch, the device 102 may calculate depth information using any method known to one of skill in the art. For example, the device 102 may calculate depth information for a particular dot using coordinates of the center dot in the captured patch and coordinates of the center dot in the corresponding reference patch. Based on these two coordinates, the device 102 may determine an approximate distance from the camera 106 to the particular dot and store the approximate distance as depth information corresponding to the particular dot.” the captured patch is considered to be the target object and the reference patch is considered to be the reference plane);
and selecting a reference image from the at least two reference images with different sizes according to the distance (see Song Col 17, Lines 8-11, “The device 102 may then select one or more reference images based on a proximity between the coordinates of the correlated dots and the coordinates of the dot from the captured image”).
The rationale of claim 4 has been applied herein.
Regarding claim 6, Song in view of Shi in further view of Gao further teaches the method according to claim 5,
wherein a size of the selected reference image is positively correlated with the distance (see Song Col 17, Lines 27-31, “To potentially improve correlation results for a third (or other) dot in the captured image neighboring the first dot, the device 102 may increase the search region associated with the first reference image or may search a second reference image taken at approximately the first depth (e.g., 5 m).”).
The rationale of claim 5 has been applied herein.
Regarding claim 7, Song in view of Shi in further view of Gao further teaches the method according to claim 4,
wherein selecting the reference image from the at least two reference images with different sizes comprises: determining a position of the target object in the target image (see Song Col 17, Lines 5-7, “the device 102 may identify coordinates of correlated dots in the reference images and may determine which coordinates are closest to coordinates of the dot from the captured image.”, identifying the coordinates of the correlated dots and determining which are closest to coordinates from the captured image is considered to be determining the position of the target object in the target image);
and determining the selected reference image from the at least two reference images with different sizes according to the position of the target object in the target image (see Song Col 17, Lines 8-11, “The device 102 may then select one or more reference images based on a proximity between the coordinates of the correlated dots and the coordinates of the dot from the captured image.”, the reference image is selected based on the proximity to coordinates in the correlated dots from the captured image which is considered to be the position of the target object).
The rationale of claim 4 has been applied herein.
Regarding claim 8, Song in view of Shi in further view of Gao further teaches the method according to claim 1,
further comprising: pre-storing the reference image in a designated storage position as a stored reference image (see Song Col 5, Lines 58-59, “the device 102 may store one or more reference images for each projected dot pattern”), and wherein generating the depth image of the target object based on the target image and the reference image comprises: reading the reference image from the designated storage position, and generating the depth image of the target object according to the target image and the stored reference image (see Song Col 5, Lines 60-64, “Thus, rather than comparing coordinates of the dots in the captured image to the coordinates of the dots in the projected dot pattern, the device 102 may compare the coordinates of the dots in the captured image to coordinates of the dots in the one or more reference images”).
The rationale of claim 1 has been applied herein.
Regarding claim 10, Song in view of Shi in further view of Gao further teaches the method according to claim 8, further comprising:
prior to matching a target pixel of the target image to a matched pixel in the reference image: performing binarization processing on the reference image and the target image (see Shi, Paragraph [0070], “the structured-light-based system may extract the structured light pattern made of the repeating patches (e.g., the structured light pattern 210), and may perform binarization (430)”).
The rationale of claim 8 has been applied herein.
Regarding claim 11, Song in view of Shi in further view of Gao further teaches the method according to claim 1,
wherein the plurality of first effective pixels is an effective pixel used when depth imaging is performed on the target object (see Song Col 4, Lines 59-63, “the device 102 may repeat the method illustrated in FIG. 1 for each of the first, second and third dot patterns to generate first, second and third depth information based on the corresponding one or more reference images.” the method in Fig. 1 is being repeated using the target in the captured images therefore it is implied the plurality of first effective images are included to obtain the target image.).
The rationale of claim 1 has been applied herein.
As per claim 12, Claim 12 claim the same limitation as Claim 1 therefore, the rejection and rationale are analogous to that made in Claim 1.
Song further teaches an electric device, comprising: one or more processors; and memory storing one or more programs, the one or more programs comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: emitting structured light to a reference plane (see Song Col 20, Lines 52-56, “The device 102 may include one or more controller(s)/processor(s) 1104 comprising one-or-more central processing units (CPUs) for processing data and computer-readable instructions, and a memory 1106 for storing data and instructions.”);
As per claim 14, Claim 14 claims the same limitation as Claim 3 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale is analogous to that made in Claim 3.
As per claim 15, Claim 15 claims the same limitation as Claim 4 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale is analogous to that made in Claim 4.
As per claim 16, Claim 16 claims the same limitation as Claim 5 and is dependent on a similarly rejected dependent claim. Therefore the rejection and rationale is analogous to that made in Claim 5.
As per claim 17, Claim 17 claims the same limitation as Claim 8 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale is analogous to that made in Claim 8.
As per claim 18, Claim 18 claim the same limitation as Claim 1 therefore, the rejection and rationale are analogous to that made in Claim 1.
Song further teaches a non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by one or more processors of an electronic device, cause the one or more processors to perform operations comprising: emitting structured light to a reference plane (Executable instructions for operating the device 102 and its various components may be executed by the controller(s)/processor(s) 1104, using the memory 1106 as temporary “working” storage at runtime. The executable instructions may be stored in a non-transitory manner in non-volatile memory 1106, storage 1108, or an external device.);
As per claim 20, Claim 20 claims the same limitation as Claim 8 and is dependent on a similarly rejected independent claim. Therefore the rejection and rationale is analogous to that made in Claim 8.
Allowable Subject Matter
Claims 21-23 are rejected under 35 U.S.C. 112(b) and objected to as being dependent upon a rejected base claim, but would be allowable if rewritten to overcome the 35 U.S.C. 112(b) rejection and in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOMINIQUE JAMES whose telephone number is (703)756-1655. The examiner can normally be reached 9:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached on (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DOMINIQUE JAMES/Examiner, Art Unit 2666 /MING Y HON/Primary Examiner, Art Unit 2666