DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the application filed on March 19, 2025. Claims 1-10 are pending. Claims 1, 7 and 8 are independent.
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on September 16, 2024 and June 30, 2025 have been considered. The submission is in compliance with the provisions of 37 CFR 1.97. The Forms PTO-1449 are signed and attached hereto.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Publication No. 2015/0302593 to Mazurenko et al. (hereinafter “Mazurenko”).
Claims 1-10 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Mazurenko.
With respect to independent claims 1 and 7, Mazurenko discloses a mask region setting unit that sets a mask region not to be subjected to smoothing processing on a 3D point cloud output from a distance measuring sensor on a basis of an intensity image output from the distance measuring sensor that measures a distance by accumulating reflected light and detecting a phase difference between irradiation light and the reflected light (see abstract: generating at least one data validity mask from the one or more image validity flag; generating at least one foreground mask from the background imagery estimation and the at least one image depth distance; generating at least one region-of-interest mask from the data validity mask and the foreground mask; generating filtered raw image data from the raw image data and at least one region of interest mask.); and
a smoothing filter processor that performs the smoothing processing on the 3D point cloud output from the distance measuring sensor and outputs the 3D point cloud, wherein the smoothing filter processor outputs the 3D point cloud of the mask region having been set as it is without performing the smoothing processing (see paragraphs [0022] and [0034]: an unfiltered depth distance image sequence from image calculation operation 302 may be denoted as d1, d2, d3 . . . dn. The background detection operation 305 may include background imagery calculation. The proposed architecture may support different background imagery calculation methods. For example, exponential smoothing may be employed where an M×N matrix imbn denotes the background imagery of an nth frame so: imb n =a·imb n−1(i,j)+(1−a)·d n(i,j) where 0<i<M, 0<j<N Eqn. 4, where imbn corresponds to the pixel at location (i,j) in a calculated background imagery and imbn−i is the same pixel in the background image from a previous frame and a is a real-valued exponential smoothing coefficient (e.g. a=0.95). Based on imbn and unfiltered depth image dn, a foreground mask E for concrete frame n may be extracted. For example, the foreground mask E(i,j) may be assigned according to whether imbn(i,j)−dn(i,j)>thr where thr is a predefined threshold (e.g. thr=10). In another embodiment, an algorithm similar to the validity mask calculation described above may be employed. A combined filtered image I may be obtained by combining the amplitude image a of unfiltered image B, the filtered depth distance image d of the filtered image C for any pixel that belongs to ROI masks G1, G2 . . . Gk, and, for all other pixels, the background estimation F. If needed, additional filtering may be applied to the combined filtered image I. For example, bilateral filtering based on depth distance image d and amplitude image a may be used. Still further, if needed, a coordinate transform (taking into account camera calibration information and known optical distortions) may be applied to convert depth distance image d to a point cloud consisting of points (xi, yi, zi) represented by 3D coordinates in a Cartesian system.).
With respect to dependent claim 2, Mazurenko discloses wherein the mask region setting unit extracts a characteristic shape from the intensity image and sets the mask region in association with a region of the characteristic shape having been extracted (see paragraphs [0002], [0025] and [0028]: generating at least one foreground mask from the background imagery estimation and the at least one image depth distance. The foreground segmentation operation 306 may return an unknown or predefined number of region-of-interest (ROI) masks G. The ROI masks G may represent different segments of the image. Scene segmentation may allow for effective noise filtering during subsequent operations. Each segment of an imaged scene may be de-noised independently from another in order to maintain the quality of edges between segments. Each ROI mask G1 . . . Gk may be an M×N matrix of values {−1, 0, 1, 2} where, again, “1” corresponds to pixels from the mask, “−1” are pixels that were removed from the mask, “0” corresponds to pixels not belonging to the mask and “2” marks the pixels that were added to an ROI mask G.).
With respect to dependent claim 3, Mazurenko discloses wherein the mask region setting unit sets the mask region in association with the region of the characteristic shape when a contrast of the region of the characteristic shape is equal to or greater than a threshold value (see paragraphs [0022] and [0028]: a calculated background imagery and imbn−i is the same pixel in the background image from a previous frame and a is a real-valued exponential smoothing coefficient (e.g. a=0.95). Based on imbn and unfiltered depth image dn, a foreground mask E for concrete frame n may be extracted. For example, the foreground mask E(i,j) may be assigned according to whether imbn(i,j)−dn(i,j)>thr where thr is a predefined threshold (e.g. thr=10). In another embodiment, an algorithm similar to the validity mask calculation described above may be employed. The output of the foreground segmentation operation 306 may be a set of ROI masks G1, G2 . . . Gk where k corresponds to the number of regions of interest found in the foreground of an image.).
With respect to dependent claim 4, Mazurenko discloses wherein the mask region setting unit sets the mask region in association with the region of the characteristic shape when a distance to the region of the characteristic shape is less than a threshold value (see paragraph [0021]: small connected sub-regions of the data validity mask D(i,j) may be removed. For example, connected sub-regions (i.e. a region (i,j) where data validity mask D(i,j)=1} may be identified and an estimated number of pixels for that region may be determined. If a given sub-region (i,j) has a size less than a given threshold value (e.g. 10 pixels), the data validity mask D(i,j) may be assigned as a first value other than 0 or 1 (e.g. −1) for each pixel of that connected sub-region.).
With respect to dependent claim 5, Mazurenko discloses further comprising a behavior determiner that sets an entry prohibition region on a basis of the 3D point cloud output from the smoothing filter processor and determines a behavior of the mobile body (see paragraphs [0033] and [0034]: Applying the filtered ROI mask G′t to the raw image data A may avoid distorting the filtering output. It may be the case that the autocorrelation raw image data A′ does not take into account the actual number of valid pixels that were used for each filtering step. bilateral filtering based on depth distance image d and amplitude image a may be used. Still further, if needed, a coordinate transform (taking into account camera calibration information and known optical distortions) may be applied to convert depth distance image d to a point cloud consisting of points (xi, yi, zi) represented by 3D coordinates in a Cartesian system.).
With respect to dependent claim 6, Mazurenko discloses wherein the distance measuring sensor includes an iToF distance measuring sensor (see paragraph [0012]: a three-dimensional (3D) data stream from a time-of-flight (ToF) camera as a source.).
With respect to independent claim 8, Mazurenko discloses a display device; an instruction device that instructs the display device to display a regular pattern; and an information processing apparatus that performs distance measurement in an environment including the display device (see paragraphs [0012] and [0019]: The proposed approach may be applicable to both configurations, providing a high image quality at a front-end output together with the low system latency. The image calculation operation 303 may be a simplified version of image calculation operation 302 where only a depth distance image d is generated and provided as a filtered image C.), wherein the information processing apparatus includes a distance measuring sensor that measure a distance by accumulating reflected light and detecting a phase difference between irradiation light and the reflected light, a mask region setting unit that extracts a characteristic shape from an intensity image output from the distance measuring sensor, and sets a mask region not to be subjected to smoothing processing on a 3D point cloud output from the distance measuring sensor in association with the region of the characteristic shape having been extracted (see abstract: generating at least one data validity mask from the one or more image validity flag; generating at least one foreground mask from the background imagery estimation and the at least one image depth distance; generating at least one region-of-interest mask from the data validity mask and the foreground mask; generating filtered raw image data from the raw image data and at least one region of interest mask.), and
a smoothing filter processor that performs the smoothing processing on the 3D point cloud output from the distance measuring sensor and outputs the 3D point cloud, and the smoothing filter processor outputs the 3D point cloud of the mask region having been set as it is without performing the smoothing processing (see paragraphs [0022] and [0034]: an unfiltered depth distance image sequence from image calculation operation 302 may be denoted as d1, d2, d3 . . . dn. The background detection operation 305 may include background imagery calculation. The proposed architecture may support different background imagery calculation methods. For example, exponential smoothing may be employed where an M×N matrix imbn denotes the background imagery of an nth frame so: imb n =a·imb n−1(i,j)+(1−a)·d n(i,j) where 0<i<M, 0<j<N Eqn. 4 where imbn corresponds to the pixel at location (i,j) in a calculated background imagery and imbn−i is the same pixel in the background image from a previous frame and a is a real-valued exponential smoothing coefficient (e.g. a=0.95). Based on imbn and unfiltered depth image dn, a foreground mask E for concrete frame n may be extracted. For example, the foreground mask E(i,j) may be assigned according to whether imbn(i,j)−dn(i,j)>thr where thr is a predefined threshold (e.g. thr=10). In another embodiment, an algorithm similar to the validity mask calculation described above may be employed. A combined filtered image I may be obtained by combining the amplitude image a of unfiltered image B, the filtered depth distance image d of the filtered image C for any pixel that belongs to ROI masks G1, G2 . . . Gk, and, for all other pixels, the background estimation F. If needed, additional filtering may be applied to the combined filtered image I. For example, bilateral filtering based on depth distance image d and amplitude image a may be used. Still further, if needed, a coordinate transform (taking into account camera calibration information and known optical distortions) may be applied to convert depth distance image d to a point cloud consisting of points (xi, yi, zi) represented by 3D coordinates in a Cartesian system.).
With respect to dependent claim 9, Mazurenko discloses wherein the instruction device includes a user terminal or a network server (see paragraph [0013]: gesture recognition system 100 including an imaging device 101, a host processing device 102 and gesture data utilization hardware 103 configured to utilize gesture data to perform one or more subsequent operations. The gesture recognition system 100 may further include a front-end interface 106 between the processing device 102 and the gesture data utilization hardware 103.).
With respect to dependent claim 10, Mazurenko discloses wherein the distance measuring sensor includes an iToF distance measuring sensor (see paragraph [0012]: a three-dimensional (3D) data stream from a time-of-flight (ToF) camera as a source.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEMETRA R SMITH-STEWART whose telephone number is (571)270-3965. The examiner can normally be reached 10am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Nolan can be reached at 571-270-7016. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEMETRA R SMITH-STEWART/Examiner, Art Unit 3661
/PETER D NOLAN/Supervisory Patent Examiner, Art Unit 3661
Prosecution Insights