Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The preliminary amendment filed on 10/08/2024 has been entered. Claims 1-15 were amended. Claims 1-15 remain pending in the application.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/27/2025 and 10/08/2024 were filed and are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Gao (US 2023/0072702, 2021).
Regarding claims 1 and 15, Gao teaches A three-dimensional 3D structured light camera (Gao, [0040], reproduced below:
PNG
media_image1.png
427
674
media_image1.png
Greyscale
. Which is being interpreted to involve 3D. As seen in [0041], “3D reconstruction”) device comprising:
an infrared IR emitting light source configured to illuminate a 3D target object (Gao, [0062], reproduced below:
PNG
media_image2.png
640
672
media_image2.png
Greyscale
. “IR sensor” is being interpreted to emit IR light that illuminates a 3D target object), a speckle emitter (Gao, [0040]: “The structured light projector 106 may be configured to generate a structured light pattern (e.g., a speckle pattern).”), and an IR camera (Gao, see [0062] image above, “IR sensor”);
a look-up table LUT (Gao, see [0033] image below: “Lookup table”) memory configured to store a reference speckle image of the 3D target object (Gao, [0033], reproduced below:
PNG
media_image3.png
460
674
media_image3.png
Greyscale
. “Reference images” of “an environment with the structured light pattern”, which is being interpreted to involve “speckle image”); and
a processor (Gao, see [0040] image above: “processor”) operably coupled to the light source (Gao, see [0062] image above, “RGB-IR sensor” is being interpreted as involving a “light source”) and the LUT memory (Gao, see [0033] image above, “lookup table”) and configured to:
perform object detection on a target speckle image (Gao, [0031], reproduced below:
PNG
media_image4.png
588
1008
media_image4.png
Greyscale
. 3D reconstruction is being interpreted to involve object detection. “speckle structured light images” is being interpreted to involve “target speckle image”);
perform sub-pixel-level SPL processing (Gao, see [0110] image below, “sub-pixels” has features that are extracted) that comprises a plurality of interpolations of the reference speckle image and the target speckle image (Gao, [0110], reproduced below:
PNG
media_image5.png
610
676
media_image5.png
Greyscale
. CNN is being interpreted as having interpolation as part of the convolution process that is done on the reference and target speckle image, see [0031, 0033] image above);
perform an adaptive binarization process (Gao, [0194], reproduced below:
PNG
media_image6.png
276
666
media_image6.png
Greyscale
. Which is being interpreted to involve adaptive binarization) wherein the adaptive binarization process searches the LUT memory for SPL speckle features of the stored reference speckle image that match SPL speckle features of the SPL target speckle image (Gao, [0206], reproduced below:
PNG
media_image7.png
434
672
media_image7.png
Greyscale
. “Speckle image block matching” is being interpreted as involving a reference and target speckle image for SPL speckle features);
find SPL speckle features of the target speckle image that match searched SPL speckle features of the stored reference speckle image (Gao, see [0206] image above, “speckle image block matching” is being interpreted to involve finding SPL speckle features, as part of the convolution techniques, for the target, used for block matching, and reference speckle image, such as in the lookup data); and
perform a depth value calculation (Gao, see [0211] image below, “depth map” is being interpreted to involve depth value calculations) of the 3D target object (Gao, see [0031] image above, “3D reconstruction” is being interpreted to involve 3D object detection) in response to the plurality of interpolations (Gao, see [0211] image below, “convolution techniques” are being interpreted to involve “interpolations”) and the found SPL speckle features of the target SPL speckle image (Gao, [0211], reproduced below:
PNG
media_image8.png
436
676
media_image8.png
Greyscale
. “Speckle image” and “convolution techniques” are being interpreted to involve SPL speckle features that are found in order to create a depth map).
Regarding claim 2, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor configured to perform the plurality of interpolations (Gao, see [0110] image below, CNN module is being interpreted to involve interpolations, which include upsampling, as seen in [0111]) of the reference SPL (Gao, see [0110] image below, “sub-pixels”) speckle image and the target SPL speckle image (Gao, see [0033] image above, “Reference images” of “an environment with the structured light pattern”, which is being interpreted to involve “speckle image” and a target SPL speckle image) comprises the processor being configured to identify pixel information between two pixels (Gao, [0110], reproduced below:
PNG
media_image9.png
606
672
media_image9.png
Greyscale
. “edges” and “shapes” are being interpreted to involve pixel information between two pixels).
Regarding claim 3, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor is further configured to identify a 3D (Gao, [0094]: “3D reconstruction”) target object area of interest (Gao, [0088]: “The processor 102 may be configured to crop (e.g., trim to) a region of interest from a full video frame (e.g., generate the region of interest video frames)”. “Region of interest” is being interpreted as “area of interest”) of the target SPL speckle image (Gao, see [0033] image above, “Reference images” of “an environment with the structured light pattern”, which is being interpreted to involve “speckle image” and a target SPL speckle image) and perform SPL processing that comprises a plurality of interpolations (Gao, see [0110] image below, CNN module is being interpreted to involve interpolations, which include upsampling, as seen in [0111]) of the reference speckle image and the target SPL speckle image (Gao, see [0033] image above, “Reference images” of “an environment with the structured light pattern”, which is being interpreted to involve “speckle image” and a target SPL speckle image) within the identified 3D target object area of interest (Gao, [0088]: “The processor 102 may be configured to crop (e.g., trim to) a region of interest from a full video frame (e.g., generate the region of interest video frames)”. “Region of interest” is being interpreted as “area of interest” in 3D).
Regarding claim 4, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor is further configured to perform a morphological operation on the extracted matched speckle features of the target speckle image that generates a noise-reduced target SPL speckle image (Gao, [0004]: “Results of 3D reconstruction using the monocular speckle structured light are affected by various factors such as the power of the speckle projector, temporal noise, spatial noise, reflectivity of the detected object, etc.”; [0081]: “The pixel data signals may be enhanced by the processor (e.g… noise filtering)”) and perform binary encoding compression (Gao, [0139]: “the preprocessing may be performed by generating a local adaptive binary expression with an adaptive offset term using convolution techniques”. Which is being interpreted to involve encoding compression as convolution techniques are involved) on the noise-reduced target SPL speckle image (Gao, [0139]: “the preprocessing may be performed by generating a local adaptive binary expression with an adaptive offset term using convolution techniques”. “Convolution techniques” are being interpreted as being applied on the noise-reduced target SPL speckle image).
Regarding claim 5, Gao teaches The 3D structured light camera device of Claim 4 wherein the processor configured to perform binary encoding compression (Gao, see [0139] image below, “local adaptive binary expression” is being interpreted as involving “binary encoding compression”) on the noise-reduced target SPL speckle image (Gao, [0004]: “Results of 3D reconstruction using the monocular speckle structured light are affected by various factors such as the power of the speckle projector, temporal noise, spatial noise, reflectivity of the detected object, etc.”; [0081]: “The pixel data signals may be enhanced by the processor (e.g… noise filtering)”. Which is being interpreted as reducing noise in the target SPL speckle image) comprises the processor being configured to compress the noise-reduced target SPL speckle image into at least one byte of image data (Gao, [0139], reproduced below:
PNG
media_image10.png
606
676
media_image10.png
Greyscale
. “Expression” is being interpreted as compressing into at least one byte of image data).
Regarding claim 6, Gao teaches The 3D structured light camera device of Claim 5 wherein the processor is configured to match (Gao, [0006]: “It would be desirable to implement accelerating speckle image block matching using convolution techniques”) at least one byte of SPL speckle features (Gao, see [0182] image below, the data used is being interpreted of having at least one byte of SPL speckle data, as seen in as an example in Figure 16 with the structured light pattern being involved with the convolution techniques) of the target SPL speckle image with searched SPL speckle features (Gao, see [0182] image below, “search the lookup data” is being interpreted to involve “searched SPL speckle features”. CNN is being interpreted as having the ability to extract features) of the stored reference SPL speckle image (Gao, [0182], reproduced below:
PNG
media_image11.png
282
674
media_image11.png
Greyscale
. The CNN module searches the lookup data is being interpreted to involve the stored reference SPL speckle image).
Regarding claim 7, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor is configured to selectively perform SPL processing (Gao, see [0134] image below, “within the range of the structured light SLP” is being interpreted as involving selectively perform SPL processing, because outside the range the SLP may not be triggered) in response to a determined distance (Gao, see [0134] image below, “within the range of the structured light SLP”. “Within the range” is being interpreted as “a determined distance”) to the 3D target object area of interest (Gao, [0134], reproduced below:
PNG
media_image12.png
770
684
media_image12.png
Greyscale
. “3D Reconstruction” is being interpreted to involve 3D target object area of interest).
Regarding claim 8, Gao teaches The 3D structured light camera device of Claim 7 wherein the processor is configured to selectively perform SPL processing (Gao, [0006]: ”It would be desirable to implement accelerating speckle image block matching using convolution techniques”. “Convolution techniques” the CNN described in the Gao reference are being interpreted as involving SPL processing) in response to a size of the 3D target object (Gao, [0088]: “The processor 102 may generate the video frames and select an area. In an example, cropping the region of interest may generate a second image. The cropped image (e.g., the region of interest video frame) may be smaller than the original video frame (e.g., the cropped image may be a portion of the captured video)”. Which shows the size of the region of interest) area of interest being below a SPL threshold target (Gao, [0089]: “The processor may update the selected region of interest coordinates and dynamically update the cropped section… based on the directional audio captured...The area of interest may change from frame to frame”. Which shows the size of the area of interest changes based on thresholds, that can include audio direction or movement because of video. For video threshold, the area of interest can change based on how far object has moved or the size of the object in the video. As an example, the size the object may change, and SPL processing may or may not change based on the size threshold, which includes below. The CNN module, see example Figure 16, selectively processes the speckle image based on the cropping, or size of the area of interest).
Regarding claim 9, Gao teaches The 3D structured light camera device of Claim 8, wherein the SPL threshold target is application dependent (Gao, [0067]: “In various embodiments, the detection of motion may be used as one threshold for activating the capture device 104”. Motion detection threshold is being interpreted as “application dependent”).
Regarding claim 10, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor configured to perform object detection on a target SPL speckle image (Gao, [0040]: “The structured light projector 106 may be configured to generate a structured light pattern (e.g., a speckle pattern)”. Which is being interpreted to involve a target SPL speckle image for object detection) comprises the processor configured to obtain one or more coordinate parameter(s) of a target bounding box (Gao, see [0089] image below, “The area of interest may be dynamically adjusted”, which is being interpreted to have one or more coordinate parameters. “Area of interest” is being interpreted to involve “a target bounding box”) from which the processor is configured to adaptively extract the 3D target object area of interest to process (Gao, [0089], reproduced below:
PNG
media_image13.png
682
1004
media_image13.png
Greyscale
. “Dynamically update the cropped section” is being interpreted as involving “adaptively extract the 3D target object area of interest to process”).
Regarding claim 11, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor is configured to convert (Gao, see [0062] image below, the monochrome mask is being interpreted to involve converting) the target SPL speckle image and the reference SPL speckle image (Gao, [0060]: “The signal VIDEO may be images comprising a background (e.g., objects and/or the environment captured) and the speckle pattern generated by the structured light projector 106”. Which is being interpreted to have target and reference speckle images) from a raw red-green-blue, RGB (Gao, [0062], reproduced below:
PNG
media_image14.png
638
672
media_image14.png
Greyscale
. “RGB sensor” is being interpreted as involving RGB image capturing from), SPL speckle image to a single-channel greyscale SPL speckle image with a pixel value range between ‘0’ to ‘255’ (Gao, see [0062] image above, “Monochrome” is being interpreted to involve greyscale image with pixel value range between ‘0’ to ‘255’).
Regarding claim 12, Gao teaches The 3D structured light camera device of Claim 1 wherein the processor being configured to perform local adaptive binarization (Gao, [0194], reproduced below:
PNG
media_image6.png
276
666
media_image6.png
Greyscale
. Which is being interpreted to involve local adaptive binarization) comprises the processor configured to separate background information and speckle SPL points of the target SPL speckle image (Gao, [0139], reproduced below:
PNG
media_image15.png
612
672
media_image15.png
Greyscale
. Which is being interpreted to involve the target SPL speckle image).
Regarding claim 13, Gao teaches The 3D structured light camera device of Claim 12 wherein the processor being configured to perform local adaptive binarization comprises the processor configured to compare and filter (Gao, see [0164] image below, “may implement a filter”) the target SPL speckle image (Gao, see [0164] image below, the CNN module is being interpreted to accept target SPL speckle image, as supported by Figure 16, as an example) using a threshold (Gao, see [0164] image below, convolution kernel and CNN module is being interpreted to use a filter based on a threshold inside the kernel window) that is applied within a set window size (Gao, [0164], reproduced below:
PNG
media_image16.png
417
512
media_image16.png
Greyscale
. Kernel size is being interpreted as a “set window size”).
Regarding claim 14, Gao teaches An integrated circuit for a 3D structured light camera device , the integrated circuit comprising a processor according to Claim 1 (Gao, [0080]: “In some embodiments, the camera system 100 may be implemented as a system on chip (SoC)”. “System on a chip” is being interpreted as involving an “integrated circuit” that also has a processor).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Nguyen et al (“3D shape, deformation, and vibration measurements using infrared Kinect sensors and digital image correlation”, 2017) discloses RGB and IR sensor with speckle image generation that uses stored reference images.
Shaung et al (“Active stereo vision three-dimensional reconstruction by RGB dot pattern projection and ray intersection”, 2021) discloses RGB speckle image generation with threshold (Figure 4).
Alhwarin et al (“IR Stereo Kinect: Improving Depth Images by Combining Structured Light with IR Stereo”, 2014) that uses RGB and IR sensors with speckle generation. Examiner respectively reminds applicant to review the entirety of the prior art cited, which includes the Kinect RGB and IR sensor, to advance prosecution.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.B.D./Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights