DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendment
Applicant submitted amendments on 7/16/2025. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Priority
Receipt is acknowledged that application is a National Stage application of PCT/JP2021/019935. Claimed priority to JP2020-119710 with a priority date of 7/13/2020 is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS(s) dated 1/6/2023 and 1/18/2024 that have been previously considered remain placed in the application file.
Overview
Claims 1-16 are pending in this application and have been considered below.
Claims 1-16 are rejected.
Applicant Arguments
In regards to Argument 1, Applicant states claims have been amended to overcome 35 USC 112(f) issues (See Remarks, page 10 top half).
In regards to Argument 2, Applicant states claims have been amended to overcome 35 USC 101 issues (See Remarks, page 10 middle).
In regards to Argument 3, Applicant states the independent claims have been amended to incorporate estimating 3D information based on “the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value”, which no reference discloses (See Remarks, pages 11-13).
Examiner’s Response
In response to Argument 1, with respect to Claim(s) 1-4, 9-11, 13-14, and 16, the Examiner has fully considered the Argument and has found it persuasive.
In response to Argument 2, with respect to Claim(s) 16, the Examiner has fully considered the Argument and has found it persuasive.
In response to Argument 3, it has been considered but is moot in view of new ground(s) of rejection based on the amendments. A new reference, Nistico, has been introduced which in Abstract discloses “estimating depth using sensor data indicative of changes in light intensity”. Nistico introduces a similar process as described in the amended claim where a pixel event is generated only if the “change in light intensity exceeds a comparator threshold” (¶3), where the threshold serves as the “first threshold value” in the claim. The events are not produced for every pixel, only specific detected positions where the luminance change meets or exceeds this threshold (¶29-33). The Examiner views this as a core concept to event-based sensors that ignore static or low-change areas to reduce computational power. Additionally, the system projects structures light patterns onto a scene, where the event sensor captures changes caused by these patterns to correlate detected events with known illumination patterns to generate mapping data. This mapping data serves as the “first depth information” only for the event positions. The sparse depth from events forms the basis for the filler 3D reconstruction. It should be noted that “three-dimensional information” as claimed includes depth information since it quantifies the z-axis distance from a reference point to objects or surfaces in a scene.
After reviewing the amendments, the Examiner interprets that Cao in view of Brandli, further in view of Nistico teaches on the amended claims that were presented. The details of the rejection are listed below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 7-16 is/are rejected under 35 U.S.C. 103 as obvious over Cao et al (US 20210304428 A1, hereafter referred to as Cao) in view of Brandli et al (US 20170241774 A1, hereafter referred to as Brandli), further in view of Nistico et al (WO 2021216479 A1, hereafter referred to as Nistico).
Claim 1
Regarding Claim 1, Cao teaches an information processing apparatus comprising: circuitry configured to
estimate first depth information based on a first detection position of irradiation light by an irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
Cao does not explicitly teach all of the first detection position being output from a first sensor position where a luminance change greater than or equal to a first threshold value has occurred, and estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value, and position and orientation information of the first sensor at each of a plurality of times.
However, Brandli teaches the first detection position being output from a first sensor configured to detect each position where a luminance change greater than or equal to a first threshold value has occurred (Brandli teaches a pixel coordinate is associated to each pixel and each pixel generated a photocurrent proportional to the intensity of the light impinging on a respective pixel, computes a signal related to a photocurrent, and each pixel outputs an address-event merely when a respective signal due to the light impinging on the respective pixel increases by an amount larger than a first threshold or decreases by an amount larger than a second threshold since a last address-event from the respective pixel, abstract, [5-9, 41-46, 79, 84-85, 102-111]), and
estimate three- dimensional information based on the first depth information (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera. [27-29, 71-78], figure 1-2 and 7-8).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao by detecting a position where a luminance change is greater than a threshold and using depth, position, and orientation information to estimate 3D information that is taught by Brandli, to make the invention that detects a luminance change related to a threshold, and uses that position, orientation and depth to estimate 3D information; thus, one of ordinary skilled in the art would be motivated to combine the references since a POSITA understands that position, orientation, and depth information is critical for accurate 3D reconstruction using irradiated light. Further, computational costs are lowered since selecting values in relation to a threshold allows the system to remove noise by selecting a certain signal out of the multiple signals obtained (Brandli, [2-4, 6-14]).
Cao in view of Brandli does not explicitly teach all of estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value.
However, Nistico teaches estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value (Nistico in Abstract discloses "each respective pixel event is generated in response to a specific pixel sensor within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold … Depth data is determined for the scene relative to a reference position based on the mapping data").
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao in view of Brandli by incorporating event-based depth estimation that is taught by Nistico, since both reference are analogous art in the field of computer vision and depth estimation; thus, one of ordinary skilled in the art would be motivated to combine the references since Cao in view of Brandli’s stereo-based object detection with Nistico’s event sensor-based depth estimation yields the predictable result of improved depth accuracy in challenging lighting conditions, thereby enhancing object recognition reliability.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 7
Regarding Claim 7, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1,
wherein the irradiator performs switching between irradiation and stop of the irradiation light (Brandli teaches the light includes a temporarily varying intensity in the form of successive light modulation patterns. A light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera, [27-29, 71-78], figure 1-2 and 7-8).
Claim 8
Regarding Claim 8, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1,
wherein the irradiator switches an irradiation pattern among a plurality of patterns different from each other (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera, Abstract, [6, 12-17,27-29, 71-78], figure 1-2 and 7-8).
Claim 9
Regarding Claim 9, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1,
wherein the circuitry estimates the first depth information based on the first detection position (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
an irradiation position of irradiation light with reference to the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]), and
a positional relationship between the first sensor and the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]).
Claim 10
Regarding Claim 10, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1,
wherein the circuitry estimates the first depth information based on the first detection position (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
a second detection position of irradiation light by the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]), and
a positional relationship between the first sensor and a second sensor (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
the second detection position being output from the second sensor configured to detect a position where a luminance change greater than or equal to a second threshold value has occurred (Brandli teaches a pixel coordinate is associated to each pixel and each pixel generated a photocurrent proportional to the intensity of the light impinging on a respective pixel, computes a signal related to a photocurrent, and each pixel outputs an address-event merely when a respective signal due to the light impinging on the respective pixel increases by an amount larger than a first threshold or decreases by an amount larger than a second threshold since a last address-event from the respective pixel, abstract, [5-9, 41-46, 79, 84-85, 102-111]).
Claim 11
Regarding Claim 11, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1,
wherein the circuitry is further configured to estimate second depth information based on a third detection position of irradiation light by the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
the third detection position being detected based on a captured image output from a third sensor (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]), and
perform integration processing based on the first depth information and the second depth information (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]).
Claim 12
Regarding Claim 12, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 11,
wherein the integration processing includes processing of estimating three-dimensional information based on the first depth information (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
the second depth information (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
position and orientation information of the first sensor (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera. [27-29, 71-78], figure 1-2 and 7-8), and
position and orientation information of the third sensor (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera. [27-29, 71-78], figure 1-2 and 7-8).
Claim 13
Regarding Claim 13, Cao in view of Brandli teaches the information processing apparatus according to claim 11,
wherein the circuitry estimates the second depth information based on the third detection position (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
an irradiation position of irradiation light with reference to the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]), and
a positional relationship between the third sensor and the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]).
Claim 14
Regarding Claim 14, Cao in view of Brandli teaches the information processing apparatus according to claim 11,
wherein the circuitry estimates the second depth information based on the third detection position and a fourth detection position of irradiation light by the irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
the fourth detection position being detected based on a captured image output from a fourth sensor (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]).
Claim 15
Regarding Claim 15, Cao teaches an information processing method comprising:
estimating first depth information based on a first detection position of irradiation light by an irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
Cao does not explicitly teach all of the first detection position being output from a first sensor configured to detect a position where a luminance change greater than or equal to a first threshold value has occurred; and estimating, by a processor, three-dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value, and position and orientation information of the first sensor at each of a plurality of times.
However, Brandli teaches the first detection position being output from a first sensor configured to detect a position where a luminance change greater than or equal to a first threshold value has occurred (Brandli teaches a pixel coordinate is associated to each pixel and each pixel generated a photocurrent proportional to the intensity of the light impinging on a respective pixel, computes a signal related to a photocurrent, and each pixel outputs an address-event merely when a respective signal due to the light impinging on the respective pixel increases by an amount larger than a first threshold or decreases by an amount larger than a second threshold since a last address-event from the respective pixel, abstract, [5-9, 41-46, 79, 84-85, 102-111]); and
estimating, by a processor, three-dimensional information based on the first depth information (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera. [27-29, 71-78], figure 1-2 and 7-8).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao by detecting a position where a luminance change is greater than a threshold and using depth, position, and orientation information to estimate 3D information that is taught by Brandli, to make the invention that detects a luminance change related to a threshold, and uses that position, orientation and depth to estimate 3D information; thus, one of ordinary skilled in the art would be motivated to combine the references since a POSITA understands that position, orientation, and depth information is critical for accurate 3D reconstruction using irradiated light. Further, computational costs are lowered since selecting values in relation to a threshold allows the system to remove noise by selecting a certain signal out of the multiple signals obtained (Brandli, [2-4, 6-14]).
Cao in view of Brandli does not explicitly teach all of estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value.
However, Nistico teaches estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value (Nistico in Abstract discloses "each respective pixel event is generated in response to a specific pixel sensor within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold … Depth data is determined for the scene relative to a reference position based on the mapping data").
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao in view of Brandli by incorporating event-based depth estimation that is taught by Nistico, since both reference are analogous art in the field of computer vision and depth estimation; thus, one of ordinary skilled in the art would be motivated to combine the references since Cao in view of Brandli’s stereo-based object detection with Nistico’s event sensor-based depth estimation yields the predictable result of improved depth accuracy in challenging lighting conditions, thereby enhancing object recognition reliability.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 16
Regarding Claim 16, Cao teaches a non-transitory computer-readable storage medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising:
estimating first depth information based on a first detection position of irradiation light by an irradiator (Cao teaches obtaining, through an infrared binocular camera, a pair of images by acquiring an image formed by a measured object under irradiation of a speckle pattern projected by a projection component. Based on a pair of images composed of the first image and the second image, using the binocular depth algorithm, the three-dimensional data of the measured object can be calculated, [7, 60-61]),
Cao does not explicitly teach all of the first detection position being output from a first sensor configured to detect a position where a luminance change greater than or equal to a first threshold value has occurred; and estimating three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value, and position and orientation information of the first sensor at each of a plurality of times.
However, Brandli teaches the first detection position being output from a first sensor configured to detect a position where a luminance change greater than or equal to a first threshold value has occurred (Brandli teaches a pixel coordinate is associated to each pixel and each pixel generated a photocurrent proportional to the intensity of the light impinging on a respective pixel, computes a signal related to a photocurrent, and each pixel outputs an address-event merely when a respective signal due to the light impinging on the respective pixel increases by an amount larger than a first threshold or decreases by an amount larger than a second threshold since a last address-event from the respective pixel, abstract, [5-9, 41-46, 79, 84-85, 102-111]); and
estimating three- dimensional information based on the first depth information (Brandli teaches a light source providing said spatially structured light and the image sensor remain in a fixed spatial position with respect to each other while moving relative to said surface so that said illuminated spatial area of said surface moves along the surface, i.e., scanning the surface. Further, according to a preferred embodiment of the method according to the invention, the pixel coordinates (u,v) of the current image of said spatial area are transformed into world coordinates (x, y, z) so as to reconstruct said surface in said world coordinates. FIG. 1 shows a setup of the optical sensor (e.g. DVS) together with a light source (e.g. line laser). (A) Schematic view of the setup. (B) photo of the DVS128 camera (i.e. optical DVS sensor having 128×128 pixel with line laser): the rigid laser mount allows a constant distance and inclination angle of the laser with respect to the camera. [27-29, 71-78], figure 1-2 and 7-8).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao by detecting a position where a luminance change is greater than a threshold and using depth, position, and orientation information to estimate 3D information that is taught by Brandli, to make the invention that detects a luminance change related to a threshold, and uses that position, orientation and depth to estimate 3D information; thus, one of ordinary skilled in the art would be motivated to combine the references since a POSITA understands that position, orientation, and depth information is critical for accurate 3D reconstruction using irradiated light. Further, computational costs are lowered since selecting values in relation to a threshold allows the system to remove noise by selecting a certain signal out of the multiple signals obtained (Brandli, [2-4, 6-14]).
Cao in view of Brandli does not explicitly teach all of estimate three-dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value.
However, Nistico teaches estimate three- dimensional information based on the first depth information estimated only for each detected position where the luminance change is greater than or equal to the first threshold value (Nistico in Abstract discloses "each respective pixel event is generated in response to a specific pixel sensor within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold … Depth data is determined for the scene relative to a reference position based on the mapping data").
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao in view of Brandli by incorporating event-based depth estimation that is taught by Nistico, since both reference are analogous art in the field of computer vision and depth estimation; thus, one of ordinary skilled in the art would be motivated to combine the references since Cao in view of Brandli’s stereo-based object detection with Nistico’s event sensor-based depth estimation yields the predictable result of improved depth accuracy in challenging lighting conditions, thereby enhancing object recognition reliability.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 2-6 is/are rejected under 35 U.S.C. 103 as obvious over Cao et al (US 20210304428 A1, hereafter referred to as Cao) in view of Brandli et al (US 20170241774 A1, hereafter referred to as Brandli), further in view of Nistico et al (WO 2021216479 A1, hereafter referred to as Nistico), further in view of Fujimoto et al (US 20150243017 A1, hereafter referred to as Fujimoto).
Claim 2
Regarding Claim 2, Cao in view of Brandli, further in view of Nistico teaches the information processing apparatus according to claim 1.
Cao in view of Brandli, further in view of Nistico does not explicitly teach all of wherein in a case where there is a disappearance point where irradiation light is not detected by the first sensor, circuitry is configured to execute predetermined processing related to the disappearance point.
However, Fujimoto teaches wherein in a case where there is a disappearance point where irradiation light is not detected by the first sensor (Fujimoto teaches FIG. 10B illustrates an example of a diagram for explaining a laser light in the case where laser light is not emitted before and after the direction to the point of disappearance. In the case where the point of disappearance is between the second direction and the third direction, the emission timing control unit 31 does not output an emission timing signal at the second timing and the third timing. Accordingly, laser light is emitted in an order of at the first timing, the point of disappearance, the fourth timing and the fifth timing. Thus, even if laser light is emitted to the point of disappearance, a time interval for emission can be spaced, and degradation of the laser diode 32 can be prevented, [81-92], figure 10B),
the integrated processing unit executes predetermined processing related to the disappearance point (Fujimoto teaches performing object recognition processing directly linked to the disappearance point, [81-92]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Cao in view of Brandli, further in view of Nistico by performing processing related to a disappearance point that is taught by Fujimoto, to make the invention that performs processing related to 3D reconstruction using a disappearance point; thus, one of ordinary skilled in the art would be motivated to combine the references since a disappearance point allows for more robust processing when handling cases where irradiation light drops off, ensuring the 3D reconstruction is reliable even with gaps in detection (Fujimoto, [2-5, 12-13, 36]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claim 3
Regarding Claim 3, Cao in view of Brandli, further in view of Nistico, further in view of Fujimoto teaches the information processing apparatus according to claim 2,
wherein the predetermined processing includes associating information indicating that a shape is indefinite with the disappearance point as the predetermined processing related to the disappearance point (Fujimoto teaches a point of disappearance where the laser light's reflected wave isn't detected because an object, like a vehicle, moves out of range. The Examiner interprets the laser radar ranging unit's detection of an absence teaches performing a task linking the disappearance point to an unrecognized or outlined object, since the shape being indefinite means it's no longer clearly defined or trackable, [81-92], figures 1A-1C, 10A-10C).
Claim 4
Regarding Claim 4, Cao in view of Brandli, further in view of Nistico, further in view of Fujimoto teaches the information processing apparatus according to claim 2,
wherein in a case where there is a detection point where irradiation light is detected at a position adjacent to the disappearance point, the predetermined processing related to the disappearance point includes complementary processing of complementing information of a shape corresponding to the disappearance point based on information of a shape corresponding to the detection point (Fujimoto teaches a laser radar ranging system where an emission timing control unit manages the emission of laser light across a sequence of timings relative to a defined point of disappearance, where the reflected wave of the laser light is no longer detected due to the horizontal movement, aligning with a disappearance point where irradiation light is not detected by a sensor, since the system explicitly detects the absence of a reflected wave. The control unit performs predetermined processing by skipping emission at the second and third timings when the disappearance point falls between them, emitting instead at adjacent timings where reflections are detected, and the ranging unit adjusts the emission strategy based on this data, which the Examiner interprets as complementary processing. When the reflected wave vanishes, the system associates an indefinite shape with the disappearance point, since the object's boundaries become unresolvable in object recognition, and uses positional data from adjacent detection points, which would include shape-related information like edges, to infer the object's movement, effectively complementing the shape ambiguity at the disappearance point. Shape related inference processing is common in positional tracking systems like for vehicle detection, [81-92], figures 1A-1C, 10A-10C).
Claim 5
Regarding Claim 5, Cao in view of Brandli, further in view of Nistico, further in view of Fujimoto teaches the information processing apparatus according to claim 4,
wherein the complementary processing includes processing of estimating an inclination of a shape corresponding to the disappearance point such that an angle formed by an inclination of a shape corresponding to the detection point and an inclination of a shape corresponding to the disappearance point is greater than or equal to a predetermined angle (Fujimoto teaches a laser radar ranging system where an emission timing control unit manages the emission of laser light across a sequence of timings relative to a defined point of disappearance, where the reflected wave of the laser light is no longer detected due to the horizontal movement, aligning with a disappearance point where irradiation light is not detected by a sensor, since the system explicitly detects the absence of a reflected wave. The control unit performs predetermined processing by skipping emission at the second and third timings when the disappearance point falls between them, emitting instead at adjacent timings where reflections are detected, and the ranging unit adjusts the emission strategy based on this data, which the Examiner interprets as complementary processing. When the reflected wave vanishes, the system associates an indefinite shape with the disappearance point, since the object's boundaries become unresolvable in object recognition, and uses positional data from adjacent detection points, which would include shape-related information like edges, to infer the object's movement, effectively complementing the shape ambiguity at the disappearance point. Shape related inference processing is common in positional tracking systems like for vehicle detection. Furthermore, the Examiner interprets this tracking process to estimate the inclination of the shape corresponding to the disappearance point, since the system's understanding of the object's movement includes not only its position but also its orientation/angle relative to the scanning direction of the laser radar. In laser radar systems designed for tracking, it is known to a POSITA that the inclination of a shape is a critical aspect of understanding the object's form and trajectory. Fujimoto's tracking of the object's horizontal movement involves estimating this inclination to maintain a good representation of the object across the scanned area, [81-92], figures 1A-1C, 10A-10C).
Claim 6
Regarding Claim 6, Cao in view of Brandli, further in view of Nistico, further in view of Fujimoto teaches the information processing apparatus according to claim 4,
wherein the complementary processing includes processing of estimating a position of a shape corresponding to the disappearance point such that a position of a shape corresponding to the detection point and a position of a shape corresponding to the disappearance point are continuous (Fujimoto teaches a laser radar ranging system where an emission timing control unit manages the emission of laser light across a sequence of timings relative to a defined point of disappearance, where the reflected wave of the laser light is no longer detected due to the horizontal movement, aligning with a disappearance point where irradiation light is not detected by a sensor, since the system explicitly detects the absence of a reflected wave. The control unit performs predetermined processing by skipping emission at the second and third timings when the disappearance point falls between them, emitting instead at adjacent timings where reflections are detected, and the ranging unit adjusts the emission strategy based on this data, which the Examiner interprets as complementary processing. When the reflected wave vanishes, the system associates an indefinite shape with the disappearance point, since the object's boundaries become unresolvable in object recognition, and uses positional data from adjacent detection points, which would include shape-related information like edges, to infer the object's movement, effectively complementing the shape ambiguity at the disappearance point. Shape related inference processing is common in positional tracking systems like for vehicle detection. Furthermore, the Examiner interprets this tracking process to estimate the inclination of the shape corresponding to the disappearance point, since the system's understanding of the object's movement includes not only its position but also its orientation/angle relative to the scanning direction of the laser radar. In laser radar systems designed for tracking, it is known to a POSITA that the inclination of a shape is a critical aspect of understanding the object's form and trajectory. Fujimoto's tracking of the object's horizontal movement involves estimating this inclination to maintain a good representation of the object across the scanned area. Fujimoto describes a scenario where the angle between the direction of the point of disappearance and the emission direction at the third timing is compared to a threshold. The system ensures that angle between the inclinations meets a minimum threshold to reflect the object's realistic motion dynamics, [81-92], figures 1A-1C, 10A-10C).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN P CASCAIS whose telephone number is (703)756-5576. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mr. O’Neal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.P.C./Examiner, Art Unit 2674
/Ross Varndell/Primary Examiner, Art Unit 2674
Date: 8/21/2025