DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/6/2026 has been entered.
Claims 1, 2, 5-11, and 16-18 are now pending. Claims 1, 5, 6, 10, and 11 have been amended. Claims 3 and 12-15 have been cancelled. New claims 16-18 have been added. Claims 1, 10, 11, and 16-18 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/22/2025, 12/23/2025, and 1/13/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claims 1, 10, and 11 are objected to because of the following minor informalities:
In the last limitation in each of claims 1, 10, and 11, replace … image, the second training data … with … image and the second training data …
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claims 16-18 recite “perform learning so as to use training data … to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image”. This imitation can be recognized as a human learning process involving observation of training data and an image, evaluation of this data and the image, and a consecutive judgment of detecting a three-dimensional object in the image based on the observations and the evaluation. Thus it falls within the “mental process” group of abstract ideas. Accordingly, each of these the claims recites an abstract idea.
This judicial exception is not integrated into a practical application because the above-indicated limitations are merely instructions to implement the abstract idea on a computer and require no more than a generic computer to perform generic computer functions. The recitation of generic computer components (“learning device” for claim 16, “a computer” for claims 17 and 18 and “a non-transitory computer-readable storage medium” … for claim 18 does not impose any meaningful limits on practicing the abstract idea. The additional element of “(training data) associating an annotation indicating a three-dimensional object with, among regions extended due to conversion of an image, which is photographed by a camera mounted in a mobile object to capture a surrounding situation of the mobile object, into a bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and a region having a single-color pattern different from a color of a road surface in the bird's eye view image” amounts to no more than adding insignificant extra-solution specifications related to the training data that does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a device, computer, and/or (non-transitory computer-readable) storage medium to perform the learning step described above amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Neither can insignificant extra-solution specifications related to the training data. Thus, none of the additional elements as generically claimed in the independent claims is sufficient to amount to significantly more than the judicial exception.
Therefore, all of these additional limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea. Thus, these independent claims are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 6, 10, 11, 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over CHIBA et al., US PGPUB 2021/0081681 Al (hereinafter as Chiba) in view of Widjaja et al., US PGPUB 2023/0046410 Al (hereinafter as Widjaja), TAKEMURA et al., US PGPUB 2016/0307054 Al (hereinafter as Takemura), and Sakamoto et al., US PGPUB 2009/0245582 A1 (hereinafter as Sakamoto).
Regarding independent claim 1, Chiba teaches a mobile object control device [see e.g. title; fig. 1; [0036]; and [0038]-[0040]] comprising a storage medium storing computer- readable commands and a processor connected to the storage medium [note the storage medium and the processor described in [0040]], the processor executing the computer-readable commands to:
acquire a subject bird's eye view image obtained by converting an image, which is photographed by a camera mounted in a mobile object to capture a surrounding situation of the mobile object, into a bird's eye view coordinate system [see e.g. [0060] indicating the conversion of a captured image to a bird’s eye view coordinate system; note the camera being mounted on the vehicle (the mobile object) as shown in fig. 5 and indicated in [0059]];
input the subject bird's eye view image to detect a three-dimensional object in the subject bird's eye view image [note the examples of 3D object types detected as described in [0061]; see also e.g. [0004]; note the object identified being output and note the input image];
detect a travelable space of the mobile object based on the detected three-dimensional object; and cause the mobile object to travel so as to pass through the travelable space [note in [0131] the grasping of an object as an obstacle in movement travel of the vehicle and accordingly supporting the smooth movement of the vehicle; note in [0050] the autonomous driving function of the vehicle as well as the control of the steering controller and the coordination among devices thus avoiding risks that may be encountered outside the vehicle].
Chiba does not explicitly teach inputting the subject bird's eye view image into a trained model, which is trained to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image, to detect a three-dimensional object in the subject bird's eye view image.
Neither does it explicitly teach that the trained model is trained based on first training data and second training data, the first training data associating an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and the second training data associating an annotation indicating a three- dimensional object with, among the regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Widjaja teaches inputting a subject bird's eye view image into a trained model, which is trained to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image, to detect a three-dimensional object in the subject bird's eye view image [see [0025] describing a machine learning model that is trained to take in sensor data and output semantic annotations representing physical features of the area; see also [0026] and [0094] and especially note that the sensor data may be a “birds-eye view” image; see also fig. 5 and [0101]-[0102]].
Widjaja further teaches that the trained model is trained based on first training data and second training data associating an annotation indicating a three-dimensional object with a region [see [0099] indicating using validated annotations 508 to train the model, as can be seen in fig. 5; note again in [0025]-[0026] that the validated annotations indicate physical objects that can be 3D objects such as cars, people, bicycles, etc.].
It would have been obvious to one of ordinary skill in the art having the teachings of Chiba and Widjaja, before the effective filing date of the claimed invention, to modify the commands taught by Chiba to explicitly specify inputting the subject bird's eye view image into a trained model, which is trained to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image, to detect a three-dimensional object in the subject bird's eye view image, and to further specify that the trained model is trained based on first training data and second training data associating an annotation indicating a three-dimensional object with a region, as per the teachings of Widjaja. The motivation for this obvious combination would be to allow for an improvement in semantic understanding for sensor data, as taught by Widjaja [see e.g. the last 4 lines of [0021] as well as [0025]-[0026]].
The previously combined art, however, does not explicitly teach that the first training data associating an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and the second training data associating an annotation indicating a three-dimensional object with, among the regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Takemura teaches the identification of a region having a radial pattern centered about a center of a lower end of the bird's eye view image, as being a distinct 3D object [see e.g. in [0093]-[0094] and [0098] the identification of a radial pattern centered about a camera in a bird’s-eye-view image; note from [0209] the lower end of the image being the closer side to the camera].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Takemura, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by Widjaja to include first and second training data associating an annotation indicating a three-dimensional object with a region, to explicitly specify that the first training data associating an annotation indicating a three- dimensional object with a region, as that taught by Widjaja, is identified as having a radial pattern centered about a center of a lower end of the bird's eye view image, as per the teachings of Takemura, and to further specify that in the bird-eye-view taught by Widjaja. The motivation for this obvious combination would be to enable precise and successful identification of real-world objects in bird’s-eye-view images which will show sides extending upward along the vertical direction in the real world as running in a radial pattern centered about a camera in the bird’s-eye-view image, as taught by Takemura [see e.g. [0094]].
The previously combined art, however, does not explicitly teach that the second training data associating an annotation indicating a three-dimensional object with, among the regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Sakamoto teaches the identification of a region having a single-color pattern different from a color of a road surface, as being a distinct object [see e.g. in [0032]-[0033] and [0008] the identification of a lane marking based on the single-color pattern of the lane being distinguished from the color of the road surface].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Sakamoto, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by Widjaja to include first and second training data associating an annotation indicating a three-dimensional object with a region, to explicitly specify that the second training data associating an annotation indicating a three- dimensional object with a region, as that taught by Widjaja, is identified as having a single-color pattern different from a color of a road surface, as per the teachings of Sakamoto, and to further specify that in the bird-eye-view taught by Widjaja. The motivation for this obvious combination would be to enable accurate distinguishing of unique entities/objects such as lane markings based on having a distinct color independent of its shape, as taught by Sakamoto [see e.g. [0033]].
Regarding claim 5, the rejection of claim 3 is incorporated.
The previously combined art further teaches that the trained model is trained based on the first training data [see the rejection of claim 1 and the teachings of Widjaja and Takemura] and third training data associating and indicating a non-three-dimensional object with a road sign in the bird's eye view image [see e.g. [0042] of Chiba indicating examples of 2D objects such as road signs, road markings, and lane markings in the images and again note the training data including associations taught by WIdiaja].
Regarding claim 6, the rejection of claim 3 is incorporated.
Widjaja further teaches that:
the processor uses an image obtained by capturing the surrounding situation of the mobile object by the camera to recognize an object included in the image, and generate a reference map in which a position of the recognized object is reflected [note in [0026] and also in [0094] the generation of a map based on sensor data such as camera images], and
wherein the processor detects the travelable space by matching the detected three-dimensional object in the subject bird's eye view image with the generated reference map [see e.g. [0068] indicating localizing the vehicle based on matching features to the reference map; note also the harmonization of camera data with bird-eye-view data described in [0097]; note the navigation (including detecting travelable space) based on the detected objects in [0025]].
It would have been obvious to one of ordinary skill in the art having the teachings of Chiba and Widjaja, before the effective filing date of the claimed invention, to modify the method taught by Chiba and modified by Widjaja, to further explicitly specify that the processor uses an image obtained by capturing the surrounding situation of the mobile object by the camera to recognize an object included in the image, and generate a reference map in which a position of the recognized object is reflected, and wherein the processor detects the travelable space by matching the detected three-dimensional object in the subject bird's eye view image with the generated reference map, as per the teachings of Widjaja. The motivation for this obvious combination would be to enable harmonizing the different inputs from the raw images and the detected features in the bird-eye-view images for proper navigation, as taught by Widjaja [see e.g. [0097].
Independent claims 10 and 11 are rejected analogously to the rejection of claim 1.
Regarding independent claim 10, Chiba also teaches a mobile object control method [see e.g. title] to be executed by a computer [see e.g. title; fig. 1; [0036]; and [0038]-[0040]], the mobile object control method comprising the operations of claim 1 (modified by the teachings of the rest of the cited art) [see the full rejection of claim 1].
Regarding independent claim 11, Chiba a non-transitory computer-readable storage medium storing a program for causing a computer [note the storage medium and the processor described in [0040]] to perform the operations of claim 1 (modified by the teachings of the rest of the cited art) [see the full rejection of claim 1].
Regarding independent claim 16, Chiba teaches a learning device configured to perform learning [see e.g. title; fig. 1; [0036]; and [0038]-[0040]] so as to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image [note the examples of 3D object types detected as described in [0061] after getting image data in a bird’s eye view image as in [0060]; see also e.g. [0004]; note the object identified being output and note the input image].
Chiba further teaches conversion of an image, which is photographed by a camera mounted in a mobile object to capture a surrounding situation of the mobile object, into the bird's eye view image [see e.g. [0060] indicating the conversion of a captured image to a bird’s eye view coordinate system; note the camera being mounted on the vehicle (the mobile object) as shown in fig. 5 and indicated in [0059]].
Chiba does not explicitly teach using training data associating an annotation indicating a three-dimensional object with, among regions extended, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Widjaja teaches using training data associating an annotation indicating a three-dimensional object with a region of a bird's eye view image [see [0099] indicating using validated annotations 508 to train the model, as can be seen in fig. 5; note again in [0025]-[0026] that the validated annotations indicate physical objects that can be 3D objects such as cars, people, bicycles, etc.; note again in [0025] describing a machine learning model that is trained to take in sensor data and output semantic annotations representing physical features of the area; see also [0026] and [0094] and especially note that the sensor data may be a “birds-eye view” image; see also fig. 5 and [0101]-[0102]].
It would have been obvious to one of ordinary skill in the art having the teachings of Chiba and Widjaja, before the effective filing date of the claimed invention, to modify the learning method taught by Chiba to explicitly specifying using training data associating an annotation indicating a three-dimensional object with a region of the bird's eye view image. as per the teachings of Widjaja. The motivation for this obvious combination would be to allow for an improvement in semantic understanding for sensor data, as taught by Widjaja [see e.g. the last 4 lines of [0021] as well as [0025]-[0026]].
The previously combined art, however, still does not explicitly teach that the training data associates an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Takemura teaches the identification of a region having a radial pattern centered about a center of a lower end of the bird's eye view image, as being a distinct 3D object [see e.g. in [0093]-[0094] and [0098] the identification of a radial pattern centered about a camera in a bird’s-eye-view image; note from [0209] the lower end of the image being the closer side to the camera].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Takemura, before the effective filing date of the claimed invention, to modify the learning method taught by Chiba and modified by Widjaja to explicitly specify that the training data associating an annotation indicating a three- dimensional object with a region, as that taught by Widjaja, is identified as having a radial pattern centered about a center of a lower end of the bird's eye view image, as per the teachings of Takemura, and to further specify that in the bird-eye-view taught by Widjaja. The motivation for this obvious combination would be to enable precise and successful identification of real-world objects in bird’s-eye-view images which will show sides extending upward along the vertical direction in the real world as running in a radial pattern centered about a camera in the bird’s-eye-view image, as taught by Takemura [see e.g. [0094]].
The previously combined art, however, still does not explicitly teach that the training data associates an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Sakamoto teaches the identification of a region having a single-color pattern different from a color of a road surface, as being a distinct object [see e.g. in [0032]-[0033] and [0008] the identification of a lane marking based on the single-color pattern of the lane being distinguished from the color of the road surface].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Sakamoto, before the effective filing date of the claimed invention, to modify the learning method taught by Chiba and modified by Widjaja to explicitly specify that the training data associating an annotation indicating a three- dimensional object with a region, as that taught by Widjaja, is identified as having a single-color pattern different from a color of a road surface, as per the teachings of Sakamoto, and to further specify that in the bird-eye-view taught by Widjaja. The motivation for this obvious combination would be to enable accurate distinguishing of unique entities/objects such as lane markings based on having a distinct color independent of its shape, as taught by Sakamoto [see e.g. [0033]].
Independent claims 17 and 18 are rejected analogously to the rejection of claim 16.
Regarding independent claim 17, Chiba also teaches a learning method [see e.g. [0004]] to be executed by a computer [see e.g. title; fig. 1; [0036]; and [0038]-[0040]], the learning method comprising the operations of claim 16 (modified by the teachings of the rest of the cited art) [see the full rejection of claim 16].
Regarding independent claim 18, Chiba a non-transitory computer-readable storage medium storing a program for causing a computer [note the storage medium and the processor described in [0040]] to perform learning following the operations of claim 16 (modified by the teachings of the rest of the cited art) [see the full rejection of claim 16].
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Chiba in view of Widjaja, Takemura, and Sakamoto, as applied to claim 1 above, and further in view of Julian et al., US PGPUB 2019/0025853 Al (hereinafter as Julian).
Regarding claim 2, the rejection of independent claim 1 is incorporated.
As above, Widjaja teaches inputting a subject bird's eye view image into a trained model, which is trained to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image, to detect a three-dimensional object in the subject bird's eye view image [see [0025] describing a machine learning model that is trained to take in sensor data and output semantic annotations representing physical features of the area; see also [0026] and especially note that the sensor data may be a “birds-eye view” image; see also fig. 5 and [0092]].
The previously combined art, however, does not explicitly teach that the trained model is trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse the three-dimensional object in the bird's eye view image.
Julian teaches a trained model that is trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse a three-dimensional object in the bird's eye view image [note in [0028] the decision whether a driver should adjust their path based on whether an object ahead of the car is in the way or not, i.e. whether the 3D object is traversable or not; see also [0096]; see also the modification of path in [0074]-[0075]; note the future path of travel being relatable to the bird’s eye view reference frame, as per the last 5 lines of [0047]].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Julian, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by the combined art, to explicitly specify that the trained model taught by Widjaja which is trained to receive input of a bird's eye view image is also trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse the three-dimensional object in the bird's eye view image, as per the teachings of Julian. The motivation for this obvious combination would be to enable a more reliable path prediction that is modifiable to avoid obstacles, as taught by Julian [see e.g. [0003]-[0006]; again see [0028]; [0074]-[0075]; and [0096]].
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Chiba in view of Widjaja, Takemura, and Sakamoto, as applied to claim 1 above, and further in view of Stein et al., US Patent No. 9,902,401 B2 (hereinafter as Stein).
Regarding claim 7, the rejection of independent claim 1 is incorporated.
Chiba further teaches that the processor uses a first subject bird's eye view image, which is obtained by converting an image capturing the surrounding situation of the mobile object by a first camera into the bird's eye view coordinate system, to detect the three-dimensional object [see e.g. [0060] indicating the conversion of a captured image to a bird’s eye view coordinate system; note the camera being mounted on the vehicle (the mobile object); note the examples of 3D object types detected as described in [0061]; see also e.g. [0004]; note the object identified being output and note the input image].
Widjaja further teaches detecting a position of an object by matching a first detected position of that object with another detected object of known positional information [again see [0068] of Widjaja indicating the use of positional data of known objects to detect positions of other object based on matchings; note also the harmonization of camera data with bird-eye-view data described in [0097]].
The previously combined art does not explicitly teach:
that the camera comprises a first camera installed at the lower part of the mobile object and a second camera installed at the upper part of the mobile object,
wherein the processor uses a second subject bird's eye view image, which is obtained by converting an image capturing the surrounding situation of the mobile object by the second camera into the bird's eye view coordinate system, to detect an object in the second subject bird's eye view image.
Stein teaches that the camera comprises a first camera installed at the lower part of the mobile object and a second camera installed at the upper part of the mobile object [see cameras 124 (lower) and 122 (upper) on fig. 2A; note col. 10, lines 46-50].
Stein further teaches the use of position information in association with detected objects [see col. 5, lines 44-48 as well as col. 15, lines 28-64].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Stein, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by the other combined art, to explicitly further specify that the camera comprises a first camera installed at the lower part of the mobile object and a second camera installed at the upper part of the mobile object, as per the teachings of Stein, wherein the processor uses a first subject bird's eye view image, which is obtained by converting an image capturing the surrounding situation of the mobile object by the first camera into the bird's eye view coordinate system, to detect the three-dimensional object, as per the teachings of Chiba, wherein the processor uses a second subject bird's eye view image, which is obtained by converting an image capturing the surrounding situation of the mobile object by the second camera into the bird's eye view coordinate system, to detect an object in the second subject bird's eye view image and position information thereof, as per the combined teachings of Chiba and Stein, and wherein the processor detects a position of the three-dimensional object by matching the detected three-dimensional object with the detected object with the position information, as per the combined teachings of Widjaja and Stein. The motivation for this obvious combination would be to enable validating the detection and location of objects by having more than one camera providing different views, as taught by Stein [see e.g. col. 10, lines 45-58].
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Chiba in view of Widjaja, Takemura, and Sakamoto, as applied to claim 1 above, and further in view of Gieske, US PGPUB 2013/0222592 Al (hereinafter as Gieske).
Regarding claim 8, the rejection of independent claim 1 is incorporated.
The previously combined art does not explicitly teach that the processor detects a hollow object shown in the image capturing the surrounding situation of the mobile object by the camera before converting the image into the bird's eye view coordinate system, and assigns identification information to the hollow object, and wherein the processor detects the travelable space further based on the identification information.
Gieske teaches the detection of a hollow object shown in an image capturing the surrounding situation of a mobile object by a camera before converting the image into a bird's eye view coordinate system, and the assignment of identification information to the hollow object, and wherein detecting a travelable space is further based on the identification information [see e.g. [0005] and [0024] –[0025] indicating the use of images capturing a moving vehicle’s surroundings, such as a tunnel or a similar structure in detecting such objects in the path of the vehicle and basing travelable space considerations upon that detection as described in [0033]-[0034]; note from [0057] that these are different from and added to any considerations from birds-eye-view images and systems].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Gieske, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by the other combined art, to explicitly specify that the processor detects a hollow object shown in the image capturing the surrounding situation of the mobile object by the camera before converting the image into the bird's eye view coordinate system, and assigns identification information to the hollow object, and wherein the processor detects the travelable space further based on the identification information, as per the teachings of Gieske. The motivation for this obvious combination would be to enable accounting for path obstacles that can be easily detected through raw camera images in identifying a travelable space for the vehicle, as taught by Gieske [see e.g. [0005]].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Chiba in view of Widjaja, Takemura, and Sakamoto, as applied to claim 1 above, and further in view of HONGO et al., US PGPUB 2008/0205706 Al (hereinafter as Hongo).
Regarding claim 9, the rejection of independent claim 1 is incorporated.
The previously combined art does not explicitly teach that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, the processor detects the same region as a three-dimensional object.
Hongo teaches that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, a processor detects the same region as a three-dimensional object [see e.g. fig. 3 and the description in [0024] and [0039]-[0043]; note from [0064] and [0067] the detection of a variation being equal to or larger than a threshold value indicating that a 3D moving object is detected; note from [0032] that height measurements are relative to a road surface].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Hongo, before the effective filing date of the claimed invention, to modify the commands taught by Chiba and modified by the other combined art, to explicitly specify that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, a processor detects the same region as a three-dimensional object, as per the teachings of Hongo. The motivation for this obvious combination would be to enable the accurate identification of 3D objects, while performing the proper adjustment between temporally consecutive bird’s-eye-view images, as taught by Hongo[see e.g. [0039]].
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 10, and 11 as well as claims 16-18 are provisionally rejected on the ground of nonstatutory double patenting as being respectively unpatentable over claims 4, 5, and 6 of copending Application No. 18/106,587 (hereinafter the reference application) in view of Takemura and Sakamoto.
Regarding independent claims 1, 10, and 11, although the claims at issue are not identical, they are not patentably distinct from each other because each of the indicated claims recites the same limitations of:
acquiring a subject bird's eye view image obtained by converting an image, which is photographed by a camera mounted in a mobile object to capture a surrounding situation of the mobile object, into a bird's eye view coordinate system [see the first limitation of each of claims 4, 5, and 6 respectively of the reference application];
inputting the subject bird's eye view image into a trained model, which is trained to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image, to detect a three-dimensional object in the subject bird's eye view image [see the second limitation of each of claims 4, 5, and 6 respectively of the reference application];
detecting a travelable space of the mobile object based on the detected three- dimensional object [see the third limitation of each of claims 4, 5, and 6 respectively of the reference application]; and
causing the mobile object to travel so as to pass through the travelable space [see the fourth and last limitation of each of claims 4, 5, and 6 respectively of the reference application] wherein the trained model is trained based on first training data and second training data, the first and second training data associating an annotation indicating a three-dimensional object [see the second limitation of claim 1 of the reference application from which each of claims 4, 5, and 6 depends].
Each of claims 4, 5, and 6 of the reference application does not explicitly claim that the first training data associates an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and the second training data associates an annotation indicating a three-dimensional object with, among the regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Takemura teaches the identification of a region having a radial pattern centered about a center of a lower end of the bird's eye view image, as being a distinct 3D object [see e.g. in [0093]-[0094] and [0098] the identification of a radial pattern centered about a camera in a bird’s-eye-view image; note from [0209] the lower end of the image being the closer side to the camera].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of Takemura, before the effective filing date of the claimed invention, to modify the training data claimed, to explicitly specify that the training data is based on first training data associating an annotation indicating the three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image, as being a distinct 3D object, as per the teachings of Takemura (after specifying it is in the bird-eye-view, as claimed). The motivation for this obvious combination would be to enable precise and successful identification of real-world objects in bird’s-eye-view images which will show sides extending upward along the vertical direction in the real world as running in a radial pattern centered about a camera in the bird’s-eye-view image, as taught by Takemura [see e.g. [0094]].
Each of claims 4, 5, and 6 of the reference application and Takemura, still does not explicitly teach that the second training data associates an annotation indicating a three-dimensional object with, among the regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Sakamoto teaches the identification of a region single-color pattern different from a color of a road surface, as being a distinct 3D object [see e.g. in [0032]-[0033] and [0008] the identification of a lane marking based on the single-color pattern of the lane being distinguished from the color of the road surface].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of Takemura and Sakamoto, before the effective filing date of the claimed invention, to modify the training data claimed, to explicitly specify that the training data is based on first training data and second training data associating an annotation indicating the three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image, as per the teachings of Sakamoto (after specifying it is in the bird-eye-view, as claimed). The motivation for this obvious combination would be to enable accurate distinguishing of unique entities/objects such as lane markings based on having a distinct color independent of its shape, as taught by Sakamoto [see e.g. [0033]].
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims, although allowed, have not been patented yet.
Regarding independent claims 16-18, although the claims at issue are not identical, they are not patentably distinct from each other because each of the indicated claims recites a learning device, learning method, and non-transitory computer-readable storage medium storing a program to perform learning [see claims 4, 5, and 6 of the reference application] so as to use training data [see the second limitation of each of claims 4, 5, and 6] associating an annotation indicating a three-dimensional object with certain regions [see the second limitation of claim 1 of the reference application from which each of claims 4, 5, and 6 depends], among regions extended due to conversion of an image, which is photographed by a camera mounted in a mobile object to capture a surrounding situation of the mobile object, into a bird's eye view image [see the first limitation of each of claims 4, 5, and 6 respectively of the reference application], to receive input of a bird's eye view image to output at least a three-dimensional object in the bird's eye view image [see the second limitation of each of claims 4, 5, and 6]].
Each of claims 4, 5, and 6 of the reference application does not explicitly claim that the training data associates an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a radial pattern centered about a center of a lower end of the bird's eye view image and a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Takemura teaches the identification of a region having a radial pattern centered about a center of a lower end of the bird's eye view image, as being a distinct 3D object [see e.g. in [0093]-[0094] and [0098] the identification of a radial pattern centered about a camera in a bird’s-eye-view image; note from [0209] the lower end of the image being the closer side to the camera].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of Takemura, before the effective filing date of the claimed invention, to modify the learning method taught by the claims to explicitly specify that the training data associating an annotation indicating a three- dimensional object with a region, as that claimed, is identified as having a radial pattern centered about a center of a lower end of the bird's eye view image, as per the teachings of Takemura, and to further specify that in the bird-eye-view claimed. The motivation for this obvious combination would be to enable precise and successful identification of real-world objects in bird’s-eye-view images which will show sides extending upward along the vertical direction in the real world as running in a radial pattern centered about a camera in the bird’s-eye-view image, as taught by Takemura [see e.g. [0094]].
Each of claims 4, 5, and 6 of the reference application and Takemura, still does not explicitly teach that the training data associates an annotation indicating a three-dimensional object with, among regions extended due to conversion into the bird's eye view image, a region having a single-color pattern different from a color of a road surface in the bird's eye view image.
Sakamoto teaches the identification of a region having a single-color pattern different from a color of a road surface, as being a distinct object [see e.g. in [0032]-[0033] and [0008] the identification of a lane marking based on the single-color pattern of the lane being distinguished from the color of the road surface].
It would have been obvious to one of ordinary skill in the art the reference application’s claims and the teachings of Takemura and Sakamoto, before the effective filing date of the claimed invention, to modify learning method taught by the claims to explicitly specify that the training data associating an annotation indicating a three- dimensional object with a region, as that claimed, is further identified as having a single-color pattern different from a color of a road surface, as per the teachings of Sakamoto, and to further specify that in the bird-eye-view claimed. The motivation for this obvious combination would be to enable accurate distinguishing of unique entities/objects such as lane markings based on having a distinct color independent of its shape, as taught by Sakamoto [see e.g. [0033]].
Claim 2 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of the reference application in view of Takemura, Sakamoto and Julian.
Claim 4 of the reference application does not explicitly claim that the trained model is trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse the three-dimensional object in the bird's eye view image.
Julian teaches a trained model that is trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse a three-dimensional object in the bird's eye view image [note in [0028] the decision whether a driver should adjust their path based on whether an object ahead of the car is in the way or not, i.e. whether the 3D object is traversable or not; see also [0096]; see also the modification of path in [0074]-[0075]; note the future path of travel being relatable to the bird’s eye view reference frame, as per the last 5 lines of [0047]].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of previously combined art and Julian, before the effective filing date of the claimed invention, to modify the claims, to explicitly specify that the trained model taught by the claims which is trained to receive input of a bird's eye view image is also trained to output information indicating whether or not the mobile object is capable of traveling so as to traverse the three-dimensional object in the bird's eye view image, as per the teachings of Julian. The motivation for this obvious combination would be to enable a more reliable path prediction that is modifiable to avoid obstacles, as taught by Julian [see e.g. [0003]-[0006]; again see [0028]; [0074]-[0075]; and [0096]].
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims, although allowed, have not been patented yet.
Claim 8 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of the reference application in view of Takemura, Sakamoto, and Gieske.
Claim 4 of the reference application does not explicitly claim that the processor detects a hollow object shown in the image capturing the surrounding situation of the mobile object by the camera before converting the image into the bird's eye view coordinate system, and assigns identification information to the hollow object, and wherein the processor detects the travelable space further based on the identification information.
Gieske teaches the detection of a hollow object shown in an image capturing the surrounding situation of a mobile object by a camera before converting the image into a bird's eye view coordinate system, and the assignment of identification information to the hollow object, and wherein detecting a travelable space is further based on the identification information [see e.g. [0005] and [0024] –[0025] indicating the use of images capturing a moving vehicle’s surroundings, such as a tunnel or a similar structure in detecting such objects in the path of the vehicle and basing travelable space considerations upon that detection as described in [0033]-[0034]; note from [0057] that these are different from and added to any considerations from birds-eye-view images and systems].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of the previously combined art and Gieske, before the effective filing date of the claimed invention, to modify the claims, to explicitly specify that the processor detects a hollow object shown in the image capturing the surrounding situation of the mobile object by the camera before converting the image into the bird's eye view coordinate system, and assigns identification information to the hollow object, and wherein the processor detects the travelable space further based on the identification information, as per the teachings of Gieske. The motivation for this obvious combination would be to enable accounting for path obstacles that can be easily detected through raw camera images in identifying a travelable space for the vehicle, as taught by Gieske [see e.g. [0005]].
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims, although allowed, have not been patented yet.
Claim 9 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of the reference application in view of Takemura, Sakamoto and Hongo.
Claim 4 of the reference application does not explicitly claim that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, the processor detects the same region as a three-dimensional object..
Hongo teaches that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, a processor detects the same region as a three-dimensional object [see e.g. fig. 3 and the description in [0024] and [0039]-[0043]; note from [0064] and [0067] the detection of a variation being equal to or larger than a threshold value indicating that a 3D moving object is detected; note from [0032] that height measurements are relative to a road surface].
It would have been obvious to one of ordinary skill in the art having the reference application’s claims and the teachings of previously combined art and Hongo, before the effective filing date of the claimed invention, to modify the claims, to explicitly specify that when a temporal variation amount of the same region in a plurality of time-series subject bird's eye view images with respect to a road surface is equal to or larger than a threshold value, the processor detects the same region as a three-dimensional object, as per the teachings of Hongo. The motivation for this obvious combination would be to enable the accurate identification of 3D objects, while performing the proper adjustment between temporally consecutive bird’s-eye-view images, as taught by Hongo [see e.g. [0039]].
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims, although allowed, have not been patented yet.
Response to Arguments
Applicant’s amendments to the claims in regards to the previously presented various informalities have been fully considered and are persuasive. These objections to the claims have been withdrawn. Applicant is however referred to the newly presented objections, as indicated above.
Applicant’s Arguments and amendments including removing the alternative phrase “or providing assistance for operation …” in the last limitation in the previously rejected independent claims served to overcome the previously presented rejections under 35 U.S.C. § 101. Accordingly, these rejections have been respectfully withdrawn. Applicant is however referred to the newly presented rejections under 35 U.S.C. § 101 for the new claims 16-18, as indicated above.
Applicant’s prior art arguments with respect to the amended independent claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. In regards to Applicant's prior art arguments with respect to the dependent claims, they are now rendered moot in view of the newly presented prior art rejections for the independent claims, as detailed above.
Examiner respectfully notes that the nonstatutory double patenting rejection is reasserted and updated, as included above, until the filling of a proper terminal disclaimer in compliance with 37 CFR 1.321(b).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Examiner notes the following cited art:
Kakinami, US PGPUB 20100134593 A1, which teaches a radial pattern centered about a center of a lower end of an image [see e.g. fig. 7].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA S AYAD whose telephone number is (571)272-2743. The examiner can normally be reached Monday-Friday, 7:30 am - 4:30 pm. Alt, Friday, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Queler can be reached at (571) 272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIA S AYAD/Primary Examiner, Art Unit 2172