DETAILED ACTION
This Non-Final Office Action is in response to claims filed 11/13/2024.
Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 11/13/2024 has been considered by the examiner.
Examiner’s Note
For readability, all claim language has been underlined.
In the prior art rejections, citations from applied references are provided at the end of each limitation in parentheses. Any further explanations that were deemed necessary by the Examiner are provided at the end of each claim limitation.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a data processing unit in claim 1,
a pose estimator in claim 4,
a semantic cloud generator in claim 4, and
a semantic local map generator in claim 4.
The data processing unit, pose estimator, semantic cloud generator, and semantic local map generator are being interpreted consistent with the Applicant’s specification in paragraph [0059] that recites “data processing unit 100 may include an embedded HW/SW system for processing the sensor data,” where “data processing unit 100 may include a pose estimator 110, a semantic cloud generator 120, and a semantic local map generator 130,” as recited in paragraph [0067], and in paragraphs [0130] through [0131] that describe the data processing unit as implemented by computing device 900 in the form of an embedded board.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5, 8-10, and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 5 and 15 recite the limitation of the continuous stereo infrared image. There is insufficient antecedent basis for this limitation in the claim. Specifically, the “stereo infrared image” in the preceding claims is not defined to be “continuous” and cannot be considered an inherent feature of the stereo infrared image.
Claim 8 is rejected under 35 U.S.C. 112(b) for incorporating the errors of claim 5 by dependency.
Claim 9 recites wherein the RGBD sensor is provided in a plurality, and wherein the plurality of RGBD sensors includes:
a first sensor and a second sensor attached to a front surface of the body of the robot to be spaced apart from each other with a predetermined distance in a direction perpendicular to a ground surface;
a third sensor and a fourth sensor attached to first and second side surfaces of the body, respectively; and
a fifth sensor attached to a rear surface of the body.’
Claim 1 establishes a singular “RGBD sensor,” and claim 9 creates a logical contradiction by attempting to redefine the “RGBD sensor” of claim 1 as a plurality of sensors. Thus, the limitations of claim 9 are indefinite for failing to further limit claim 1 by attempting to change the definition of the hardware (i.e. “RGBD sensor”). For example, amending the limitation of “a RGBD sensor” in claim 1 to recite “at least one RGBD sensor,” such that claim 9 recites “wherein the at least one RGBD sensor includes a first sensor and a second sensor…, a third sensor and a fourth sensor,…and a fifth sensor…” would resolve this issue.
Claim 10 is rejected under 35 U.S.C. 112(b) for incorporating the errors of claim 9 by dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 11, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jiao et al. (“Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments,” August 1, 2024, IEEE), hereinafter Jiao.
Claim 1
Jiao discloses the claimed semantic local map generation device (see Figure 4a, depicting a mapping device), comprising:
a multi-sensor unit including a RGB sensor and an inertial measurement unit (IMU) sensor attached to a body of a robot (see section III(A), regarding the sensor configuration that includes IMU, LiDAR, and RGB cameras; section V(A), regarding the mapping device includes LiDAR, two global-shutter color cameras, and one IMU installed on an autonomous vehicle, as depicted in Figure 4); and
a data processing unit operatively connected to the multi-sensor unit (see Figure 2, depicting the mapping system that receives input from the sensors) and configured for estimating a pose of the robot (see section IV(A), regarding that state estimator utilizes LiDAR-visual-IMU (LVI) odometry for real-time pose estimation) and a semantic point cloud with respect to a driving region from sensor data obtained from the multi-sensor unit (see section IV(C)(3), regarding that each voxel is projected onto the image plane to obtain the corresponding semantic label, where the voxels are determined from the undistorted point cloud detected by LiDAR, as described in (1) and (2)) and generate a semantic local map based on the estimated pose and the estimated semantic point cloud (see section IV(C)(3), regarding that the global metric-semantic mesh is extracted using the projected voxels and their associated probability distribution; section I(C)(3), with respect to Figure 1, regarding the 3D global mesh of environments is constructed using sensors’ measurements and poses as input; Figure 2, depicting the “semantic mapping” for the construction of a global metric-semantic mesh map). Given that the global metric-semantic map of Jiao is generated based on real-time sensor data representative of the current environment of the robot (see section I(C)), the global metric-semantic map of Jiao may be reasonably interpreted as “local.” The term “global” is used in Jiao due to the persistent storage of updates as the robot drives around a campus (see Figure 5).
As discussed in section III(A), Jiao discloses using a RGB sensor as an alternative to a RGB-D sensor, and therefore, Jiao does not particularly disclose the “multi-sensor unit” as including a RGBD sensor, in light of the overall claim. However, modifying the application of Jiao to be indoor environments would be obvious to one of ordinary skill in the art.
Specifically, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the RGB sensor of Jiao to be substituted with a RGBD sensor, in light of alternative embodiments described Jiao, with the predictable result of reducing the processing requirements by using the depth information of RGB-D sensors instead of performing external processing for 3D structure recovery in environments that are not limited by distance and lighting variations (section III(A) of Jiao).
Claims 3 and 13
Jiao further discloses that the data processing unit is further configured to transfer information on driveable and undriveable regions to the robot in a form of a set of 3-dimensional coordinates by use of the generated semantic local map (see section I(C)(3)-(4), regarding that the 3D global mesh is used for identifying drivable areas by analyzing the geometric and semantic attributes of the resulting mesh map; section IV(D), regarding the identification of drivable “road” regions from regions that are not, e.g., “sidewalk” and “grass;” section IV(C)(1)-(2), regarding that each voxel is associated with a three dimensional coordinate of its center, where the voxels are used to generate the global metric-semantic mesh, as described in (3)).
Claim 11
Jiao discloses the claimed semantic local map generation method (see Figure 2), comprising:
obtaining sensor data for generating a semantic local map through multi-sensors including an RGB sensor and an inertial measurement unit (IMU) sensor mounted on a body of a robot (see section III(A), regarding that the global metric-semantic map is constructed using data provided from a sensor configuration that includes IMU, LiDAR, and RGB cameras; section V(A), regarding the mapping device includes LiDAR, two global-shutter color cameras, and one IMU installed on an autonomous vehicle, as depicted in Figure 4); and
estimating a pose of the robot (see section IV(A), regarding that state estimator utilizes LiDAR-visual-IMU (LVI) odometry for real-time pose estimation) and a semantic point cloud from the sensor data (see section IV(C)(3), regarding that each voxel is projected onto the image plane to obtain the corresponding semantic label, where the voxels are determined from the undistorted point cloud detected by LiDAR, as described in (1) and (2)), and generating the semantic local map in a 3-dimensional world coordinate system by use of the estimated pose and the estimated semantic point cloud (see section IV(C)(3), regarding that the global metric-semantic mesh is extracted using the projected voxels and their associated probability distribution; section I(C)(3), with respect to Figure 1, regarding the 3D global mesh of environments is constructed using sensors’ measurements and poses as input; Figure 2, depicting the “semantic mapping” for the construction of a global metric-semantic mesh map; section IV(D), regarding that the resulting map is in the world frame, as defined in section III(B)). Given that the global metric-semantic map of Jiao is generated based on real-time sensor data representative of the current environment of the robot (see section I(C)), the global metric-semantic map of Jiao may be reasonably interpreted as “local.” The term “global” is used in Jiao due to the persistent storage of updates as the robot drives around a campus (see Figure 5).
As discussed in section III(A), Jiao discloses using a RGB sensor as an alternative to a RGB-D sensor, and therefore, Jiao does not particularly disclose the “multi-sensor unit” as including a RGBD sensor, in light of the overall claim. However, modifying the application of Jiao to be indoor environments would be obvious to one of ordinary skill in the art.
Specifically, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the RGB sensor of Jiao to be substituted with a RGBD sensor, in light of alternative embodiments described Jiao, with the predictable result of reducing the processing requirements by using the depth information of RGB-D sensors instead of performing external processing for 3D structure recovery in environments that are not limited by distance and lighting variations (section III(A) of Jiao).
Claims 2, 4-8, 12, and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Jiao in view of Alhwarin et al. (“IR Stereo Kinect: Improving Depth Images by Combining Structured Light with IR Stereo,” 2014, PRICAI 2014), hereinafter Alhwarin.
Claims 2 and 12
While Jiao further discloses that the multi-sensor unit is configured to obtain the sensor data including an image, IMU data, RGB data, and depth data (see section III(A), regarding that data from an IMU, RGB images from cameras, and 3D point clouds provided from LiDAR, which are projected onto a depth and height image, as described in section IV(C)(1)), Jiao does not further disclose that the image is a stereo infrared (IR) image. However, in modifying the RGB sensor of Jiao to be an RGB-D sensor, as discussed in the rejection of claim 1, it would be obvious to include a stereo IR image in the sensor data of Jiao, in light of Alhwarin.
Specifically, Alhwarin teaches the known technique of modifying an RGB-D sensor (similar to the RGBD sensor taught by Jiao) to be provided as a stereo RGB-D camera systems for generating a stereo infrared (IR) image (see abstract, regarding the IR images of the stereo pair of RGB-D sensors are used to generate a depth map).
Since the systems of Jiao and Alhwarin are directed to the same purpose, i.e. obtaining an image from sensors that include an RGBD sensor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the RGBD sensor taught by Jiao to be provided as a stereo RGBD system, so as to obtain the sensor data including a stereo infrared (IR) image, in light of Alhwarin, with the predictable result of providing better depth image results than a single RGB-D camera alone (last paragraph on page 410 of Alhwarin).
Claims 4 and 14
Jiao, as modified by Alhwarin, further discloses that the data processing unit includes a pose estimator configured for estimating the pose of the robot based on the stereo infrared image and the IMU data (see section IV(A), regarding that state estimator utilizes LiDAR-visual-IMU (LVI) odometry for real-time pose estimation, where LVI receives data from the IMU, LiDAR, and RGB cameras), where the image data of the RGB cameras of Jiao is modified by Alhwarin to teach the “stereo infrared image,” as discussed in the rejections of claims 2 and 12.
Jiao further discloses that the data processing unit includes:
a semantic cloud generator configured to generate the semantic point cloud based on the RGB data and the depth data (see section IV(C)(3), regarding that each voxel is projected onto the image plane to obtain the corresponding semantic label, where the voxels are determined from the undistorted point cloud detected by LiDAR, as described in (1) and (2), where the image plane is obtained from LiDAR/RGB-D scans, as described in the first paragraph of section IV(C)); and
a semantic local map generator configured to generate the semantic local map based on the pose and the semantic point cloud (see section IV(C)(3), regarding that the global metric-semantic mesh is extracted using the projected voxels and their associated probability distribution; section I(C)(3), with respect to Figure 1, regarding the 3D global mesh of environments is constructed using sensors’ measurements and poses as input; Figure 2, depicting the “semantic mapping” for the construction of a global metric-semantic mesh map).
Claims 5 and 15
Jiao, as modified by Alhwarin, further discloses that the pose estimator is configured for estimating a position, a direction, and a speed of the robot by use of matching between visual feature points obtained from the continuous stereo infrared image and a pre-integration result of the IMU data (see section IV(A), with respect to Figure 2, regarding state estimator uses LVI odometry for real-time pose estimation, which integrates LIO and VIO, where LIO uses IMU measurements for motion propagation, and VIO renders a 3D map with RGB input images, where the IMU provides high-rate linear acceleration and angular velocity measurements, as described in section III(A)). Given no integration of the IMU data is claimed, the limitation of “pre-integration result” is interpreted under its broadest reasonable interpretation.
Claims 6 and 16
Jiao further discloses that the semantic cloud generator is configured to generate a semantic image through the RGB data (see section III(A), regarding that the semantic segmentation is achieved through camera data for pixel-wise object labeling), and generate the semantic point cloud of a 3-dimensional (3D) coordinate system reference from a 2-dimensional (2D) image coordinate system of a sensor origin by use of the depth data and an intrinsic parameter of a camera (see section I(C)(3), regarding that the 3D global mesh is constructed from the 2D pixel-wise segmentation, where the voxels are determined from the undistorted point cloud detected by LiDAR, as described in (1) and (2), and camera intrinsics are calibrated, as described in section III(C)).
Claims 7 and 17
Jiao further discloses that the semantic cloud generator is configured to determine a 2-dimensional semantic image from the RGB data (see section III(A), regarding that the semantic segmentation is achieved through camera data for pixel-wise object labeling), and determine a 3-dimensional semantic point cloud by combining the depth data and the 2-dimensional semantic image (see section I(C)(3), regarding that the 3D global mesh is constructed from the 2D pixel-wise segmentation; section IV(C)(3), regarding that each voxel is projected onto the image plane to obtain the corresponding semantic label, where the voxels are determined from the undistorted point cloud detected by LiDAR, as described in (1) and (2)).
Claims 8 and 18
Jiao further discloses that the semantic local map generator is configured to generate the semantic local map in a 3-dimensional world coordinate system by multiplying the pose and a 3-dimensional semantic point cloud (see Figure 2, depicting the poses generated by state estimator are applied to the depth and height image, generated from a projected point cloud, as described in IV(C)(1), to generate the metric-semantic mesh map).
Claim 19
Jiao further discloses that a field of view (FOV) of the RGB data is smaller than a FOV of the depth data (see section IV(C), regarding that the LiDAR has a large FOV in comparison to the limited FOV of the cameras, as described in section IV(D)). Therefore, Jiao further discloses that the generating of the semantic local map includes extracting a maximum FOV by combining depth data of a region outside the FOV of the RGB with the RGB data (see section I(C), regarding that metric-semantic mapping takes sensors’ measurements and poses as input for constructing the 3D global mesh, where the sensors include LiDAR and cameras, described in section III(A)). Given that both LiDAR and cameras with respective FOVs are used in the generation of the 3D global mesh in Jiao, the combined sensor data may inherently represent a maximum FOV.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Jiao in view of Tsukamoto et al. (US 2018/0073214 A1), hereinafter Tsukamoto.
Claim 9
Jiao further discloses that the RGBD sensor is provided in a plurality (see section III(A)), Jiao does not further disclose that the plurality of RGBD sensors includes:
a first sensor and a second sensor attached to a front surface of the body of the robot to be spaced apart from each other with a predetermined distance in a direction perpendicular to a ground surface;
a third sensor and a fourth sensor attached to first and second side surfaces of the body, respectively; and
a fifth sensor attached to a rear surface of the body.
However, the technique of mounting a plurality of sensors around a vehicle is well-known in the art and would be obvious to incorporate into the robot of Jiao, in light of Tsukamoto.
Specifically, Tsukamoto teaches a plurality of cameras 41-46 (similar to the RGBD sensor taught by Jiao) that include a first sensor (i.e. middle front camera 41) and a second sensor (i.e. left front camera 45) attached to a front surface of the body of bulldozer 1 (similar to the robot taught by Jiao) to be spaced apart from each other with a predetermined distance in a direction perpendicular to a ground surface (see Figure 9, depicting middle front camera 41 positioned below left front camera 45), a third sensor (i.e. left side camera 43) and a fourth sensor (i.e. right side camera 44) attached to first and second side surfaces of the body, respectively (see Figure 5, depicting left side camera 43 and right side camera 44 on bulldozer 1), and a fifth sensor (i.e. rear camera 42) attached to a rear surface of the body (see ¶0040-0041, with respect to Figure 1, depicting rear camera 42).
In Jiao, a plurality of sensors are mounted on an autonomous vehicle (i.e. “robot”). In Tsukamoto, a plurality of sensors are mounted on a bulldozer. However, it is the mounting locations of the plurality of sensors on a vehicle that is modified by Tsukamoto; therefore, the type of vehicle does not influence this combination.
Since the systems of Jiao and Tsukamoto are directed to the same purpose, i.e. using cameras to capture the environment of a vehicle, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the robot of Jiao to include a first sensor and a second sensor attached to a front surface of the body of the robot to be spaced apart from each other with a predetermined distance in a direction perpendicular to a ground surface, a third sensor and a fourth sensor attached to first and second side surfaces of the body, respectively, and a fifth sensor attached to a rear surface of the body, in light of Tsukamoto, with the predictable result of improving visibility by capturing images of the surrounding of the vehicle (¶0040 of Tsukamoto) for easily recognizing the traveling state (¶0011 of Tsukamoto).
Claim 10
Tsukamoto further teaches that the first sensor and the fifth sensor are in parallel to the ground surface (see Figure 3, depicting optical axis Ax1 of middle front camera 41 that extends toward the front and optical axis Ax2 of rear camera 42 that extends to the rear, as described in ¶0041), wherein the second sensor is attached to be closer to the ground surface than the first sensor, and tilted in a direction toward the ground surface (see ¶0049, with respect to Figure 2, regarding the optical axis of Ax5 of left front camera 45 is inclined toward the front and downward), and wherein the third sensor and the fourth sensor are tilted in the direction toward the ground surface (see ¶0044, with respect to Figure 5, regarding optical axis Ax3 of left side camera 43 is inclined to the left and downward, and optical axis Ax4 of right side camera 44 is inclined to the right and downward). Tsukamoto does not clearly depict whether the left side camera 43 and right side camera 44 are rotated toward the front surface. However, given that Tsukamoto discloses various modifications of the side cameras (see ¶0058-0061) and is directed to better understanding the progress of work and traveling state (see ¶0006), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the third and fourth sensors of Tsukamoto, so as to be rotated toward the front surface, with the predictable result of modifying the field of views of the side cameras to be focused on the progress of work and travelling state.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Jiao in view of Levinson et al. (US 9,612,123 B1), hereinafter Levinson.
Claim 20
While Jiao further discloses that the semantic local map displays an obstacle, a driveable region, and a undesignated region, in a manner to be distinguished from each other in the 3-dimensional world coordinate system (see Figure 5, depicting an example of the 3D global mesh with annotations representative of various “obstacles” and “driveable region;” Figure 6, depicting additional annotations such as “car” and areas that are not labeled that may be interpreted as “undesignated”), Jiao does not further disclose the display of a person. However, it would be capable of instant and unquestionable demonstration to incorporate a “person” annotation into the plurality of annotations taught by Jiao, in light of Levinson.
Specifically, Levinson teaches the known technique of associating semantic labels with sensor data, such as Lidar data 346a and camera data 340a (similar to the semantic local map taught by Jiao) that displays a person (see col. 18, lines 43-63, regarding that dynamic objects 1013 such as pedestrians, roadways, and other objects are displayed, where external objects are classified as pedestrians using semantic labels).
Since the systems of Jiao and Levinson are directed to the same purpose, i.e. associating a detected environment with semantic labels, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the semantic local map of Jiao to further display a person, in light of Levinson, with the predictable result of incorporating the ability to identify additional external objects, such as pedestrians (col. 18, lines 43-63 of Levinson), that would be applicable to the outdoor environment of Jiao that includes sidewalks commonly used by pedestrians (Figure 5 of Jiao).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Specifically, Motoyama (US 2021/0019536 A1) teaches generating an environment map based on a distance image generated from three-dimensional point cloud information detected by a depth sensor and an image captured by a camera (see ¶0040-0045, with respect to Figure 1), Chiu et al. (US 2020/0184718 A1) teaches creating a three-dimensional navigation map in real-time by fusing a 3D point cloud and a 2D image (see ¶0027), and Shen et al. (US 2025/0290770 A1) teaches creating an HD map by labeling fused data collected from a plurality of sensors (see ¶0021).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sara J Lewandroski whose telephone number is (571)270-7766. The examiner can normally be reached Monday-Friday, 9 am-5 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramya P Burgess can be reached at (571)272-6011. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SARA J LEWANDROSKI/Examiner, Art Unit 3661