DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The amendment filed on April 01, 2026 has been entered. Claims 1-5, 8-18, and 20 have been
amended. Claim 21 is new. Claims 6 and 7 are cancelled. The remaining claims are in original or previously presented form. Therefore, claims 1-5 and 8-21 are pending in the application.
Applicant's arguments, see applicant’s Remarks for U.S.C. § 103, filed on 04/01/2026 regarding
U.S.C. § 103 rejections have been fully considered but they are not persuasive.
Applicant’s arguments with respect to claims 1-5 and 8-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) (hereinafter Yasui) in view of Yang (CN 110456796 A).
Regarding claim 1, Yasui teaches an image processing method(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132), performed by an electronic device(Yasui, paragraph 11, image processing unit using the image obtaining device), comprising:
acquiring a road image collected by an image collection apparatus installed on a vehicle(The specification discloses an image collection apparatus can be a camera. Yasui, paragraph 32, The camera 10 is mounted on any part of the vehicle (hereinafter referred to as the vehicle M) carrying the vehicle system 1. Yasui, paragraph 81, the image obtained from the camera 10 identifies the road surface area and the road area in the first embodiment obtained from the transformation of the eye-view recognition); detecting, based on the road image, a plurality of road boundaries in the road image(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determining a target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries(The specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building). wherein
the determining the target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries is determining from the plurality of road boundaries(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Furthermore, the present specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), the target road boundary dangerous to the vehicle, based on road information determined by the road image(Similar to the target road boundary that is dangerous based on obstacle information, Yasui discloses image processing to determine a road area that is shielded by an obstacle such as a building which can be a target boundary that is dangerous. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), wherein
the determining from the plurality of road boundaries, the target road boundary dangerous to the vehicle, based on the road information determined by the road image(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
determining, based on the road information, a real road region(Yasui discloses extracting road area that includes stop line, division line that is similar to the real road region where a vehicle can drive. Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like.) and an unknown region which is unidentifiable by the vehicle(According to the present specification, unknown region may be a region obscured by an obstacle. Yasui discloses identifying a shielded area, that is similar to unknown region shielded by an object. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building);
determining, based on the real road region and the unknown region, a road boundary invisible to the vehicle(According to the specification, a road boundary, which cannot be sensed by the driver or sensor in the vehicle because of being obscured by the building can be referred to as an invisible road boundary. Similarly, Yasui’s image processing unit detects an unidentified area that is obscured by a building representing an invisible road boundary. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determining the road boundary invisible to the vehicle as the target road boundary(As discussed above, similar to the specification where it discloses the invisible area as being the road boundary obscured by the building, Yasui discloses a region where it determines an unidentified area obscured by an obstacle such as a building. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), and wherein
the determining, based on the real road region and the unknown region, the road boundary invisible to the vehicle(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
converting a collection viewpoint of the real road region and a collection viewpoint of the unknown region into a bird's eye view(Yasui discloses generating an aerial view, similar to bird’s eye view, to determine road area and shading area, which is similar to real road region and unknown region respectively. Yasui’s shading area is similar to the unknown region as it a region obscured by an obstacle such as building. Yasui, Paragraph 48, the image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area), to obtain a converted real road region(Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner) and a converted unknown region(Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area);
While Yasui teaches the identification of road lanes and boundary based on the image data, it fails to disclose determining an overlapping region of the converted real road region and the converted unknown region; and determining a road boundary in the overlapping region as the road boundary invisible to the vehicle.
However, Yang, which is in the same analogous art and that teaches about visual blind area detection discloses determining an overlapping region of the converted real road region and the converted unknown region(Yang discloses determining the intersection(overlapping)region between an extended boundary of a blind spot and road region. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone); and
determining a road boundary in the overlapping region as the road boundary invisible to the vehicle(As discussed above, Yang discloses extended boundary of a blind spot which indicates an invisible region. Yang, paragraph 20, extending each boundary of the blind spot outward by a preset distance, determining a first area based on the extended boundaries, determining the intersection of the first area and the road area, and using the intersection as a warning area. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui with Yang to determine an intersection between blind spot(invisible) region and real road region. By determining the intersection region, it is possible to warn a driver of a danger that could cause an accident.( Yang, paragraph 29, extend each boundary of the visual blind spot by a preset distance, determine a first area based on the extended boundary, and determine the intersection of the first area and the road area, and use the intersection as a warning area).
Regarding claim 2, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone ), wherein the detecting, based on the road image, the plurality of road boundaries in the road image comprises:
detecting the road image to determine a plurality of road boundaries associated with the vehicle(Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface).
Regarding claim 3, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the detecting, based on the road image, the plurality of road boundaries in the road image comprises:
detecting the road image to obtain a plurality of lanes in the road image(Yasui, paragraph 46, The recognition unit 130 identifies, for example, the lane (driving lane) in which the vehicle M is currently traveling. For example, the recognition unit 130 compares the pattern of road markings (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings surrounding the vehicle M identified from the image captured by the camera 10, thereby identifying the driving lane); and
connecting respective ends of the plurality of lanes to obtain the plurality of road boundaries( Yasui discloses identifying boundaries and lanes based on road dividing line and guardrail etc. indicating its capability to connect diving lines or lanes to construct road boundaries. Yasui, paragraph, Yasui, paragraph 46, The recognition unit 130 identifies, for example, the lane (driving lane) in which the vehicle M is currently traveling. For example, the recognition unit 130 compares the pattern of road markings (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings surrounding the vehicle M identified from the image captured by the camera 10, thereby identifying the driving lane. The recognition unit 130 is not limited to recognizing road markings; it can also recognize driving lane boundaries (road boundaries) including road markings, shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results from the INS can also be incorporated).
Regarding claim 18, Yasui teaches an image processing apparatus(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132), comprising a memory for storing instructions(Yasui, paragraph 54, memory (not shown) stores the information)and a processor, wherein the processor is configured to execute the instructions to(Yasui, paragraph 102, The hardware processor performs the following processing by executing the program stored in the storage device):
acquire a road image collected by an image collection apparatus installed on a vehicle(The specification discloses an image collection apparatus can be a camera. Yasui, paragraph 32, The camera 10 is mounted on any part of the vehicle (hereinafter referred to as the vehicle M) carrying the vehicle system 1. Yasui, paragraph 81, the image obtained from the camera 10 identifies the road surface area and the road area in the first embodiment obtained from the transformation of the eye-view recognition);
detect, based on the road image, a plurality of road boundaries in the road image(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determine a target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries(The specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building). wherein
the determining the target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries is determining from the plurality of road boundaries(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Furthermore, the present specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), the target road boundary dangerous to the vehicle, based on road information determined by the road image(Similar to the target road boundary that is dangerous based on obstacle information, Yasui discloses image processing to determine a road area that is shielded by an obstacle such as a building which can be a target boundary that is dangerous. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), wherein
the determining from the plurality of road boundaries, the target road boundary dangerous to the vehicle, based on the road information determined by the road image(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
determining, based on the road information, a real road region(Yasui discloses extracting road area that includes stop line, division line that is similar to the real road region where a vehicle can drive. Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like)and an unknown region which is unidentifiable by the vehicle(According to the present specification, unknown region may be a region obscured by an obstacle. Yasui discloses identifying a shielded area, that is similar to unknown region shielded by an object. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building);
determining, based on the real road region and the unknown region, a road boundary invisible to the vehicle(According to the specification, a road boundary, which cannot be sensed by the driver or sensor in the vehicle because of being obscured by the building can be referred to as an invisible road boundary. Similarly, Yasui’s image processing unit detects an unidentified area that is obscured by a building representing an invisible road boundary. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determining the road boundary invisible to the vehicle as the target road boundary(As discussed above, similar to the specification where it discloses the invisible area as being the road boundary obscured by the building, Yasui discloses a region where it determines an unidentified area obscured by an obstacle such as a building. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), and wherein
the determining, based on the real road region and the unknown region, the road boundary invisible to the vehicle(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
converting a collection viewpoint of the real road region and a collection viewpoint of the unknown region into a bird's eye view(Yasui discloses generating an aerial view, similar to bird’s eye view, to determine road area and shading area, which is similar to real road region and unknown region respectively. Yasui’s shading area is similar to the unknown region as it a region obscured by an obstacle such as building. Yasui, Paragraph 48, the image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area), to obtain a converted real road region(Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner) and a converted unknown region(Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area);
While Yasui teaches the identification of road lanes and boundary based on the image data, it fails to disclose determining an overlapping region of the converted real road region and the converted unknown region; and determining a road boundary in the overlapping region as the road boundary invisible to the vehicle.
However, Yang, which is in the same analogous art and that teaches about visual blind area detection discloses determining an overlapping region of the converted real road region and the converted unknown region(Yang discloses determining the intersection(overlapping)region between an extended boundary of a blind spot and road region. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone); and determining a road boundary in the overlapping region as the road boundary invisible to the vehicle(As discussed above, Yang discloses extended boundary of a blind spot which indicates an invisible region. Yang, paragraph 20, extending each boundary of the blind spot outward by a preset distance, determining a first area based on the extended boundaries, determining the intersection of the first area and the road area, and using the intersection as a warning area. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui with Yang to determine an intersection between blind spot(invisible) region and real road region. By determining the intersection region, it is possible to warn a driver of a danger that could cause an accident.( Yang, paragraph 29, extend each boundary of the visual blind spot by a preset distance, determine a first area based on the extended boundary, and determine the intersection of the first area and the road area, and use the intersection as a warning area).
Regarding claim 19, the combination of Yasui and Yang teaches the image processing apparatus of claim 18(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein when detecting, based on the road image, the plurality of road boundaries in the road image, the processor is further configured to execute the instructions to:
detect the road image to determine a plurality of road boundaries associated with the vehicle(Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface).
Regarding claim 20, Yasui teaches a non-transitory computer storage medium having stored thereon computer-executable instructions that(Yasui, paragraph 54, memory (not shown) stores the information; Yasui, paragraph 102, hardware processor), when executed, are capable of implementing operations of(Yasui, paragraph 102, The hardware processor performs the following processing by executing the program stored in the storage device):
acquiring a road image collected by an image collection apparatus installed on a vehicle(The specification discloses an image collection apparatus can be a camera. Yasui, paragraph 32, The camera 10 is mounted on any part of the vehicle (hereinafter referred to as the vehicle M) carrying the vehicle system 1. Yasui, paragraph 81, the image obtained from the camera 10 identifies the road surface area and the road area in the first embodiment obtained from the transformation of the eye-view recognition);
detecting, based on the road image, a plurality of road boundaries in the road image(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determining a target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries(The specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building). wherein
the determining the target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries is determining from the plurality of road boundaries(Yasui discloses detecting multiple road boundaries such road area(non-obstacle area) and occlusion area(obstacle area) indicating its capability to identify plurality of boundaries. Furthermore, the present specification discloses a road boundary obscured by an obstacle can be considered dangerous. Similarly, Yasui teaches determining shielded area concealed by an obstacle which can be a target road boundary dangerous to a vehicle Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), the target road boundary dangerous to the vehicle, based on road information determined by the road image(Similar to the target road boundary that is dangerous based on obstacle information, Yasui discloses image processing to determine a road area that is shielded by an obstacle such as a building which can be a target boundary that is dangerous. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), wherein
the determining from the plurality of road boundaries, the target road boundary dangerous to the vehicle, based on the road information determined by the road image(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
determining, based on the road information, a real road region(Yasui discloses extracting road area that includes stop line, division line that is similar to the real road region where a vehicle can drive. Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like) and an unknown region which is unidentifiable by the vehicle(According to the present specification, unknown region may be a region obscured by an obstacle. Yasui discloses identifying a shielded area, that is similar to unknown region shielded by an object. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building);
determining, based on the real road region and the unknown region, a road boundary invisible to the vehicle(According to the specification, a road boundary, which cannot be sensed by the driver or sensor in the vehicle because of being obscured by the building can be referred to as an invisible road boundary. Similarly, Yasui’s image processing unit detects an unidentified area that is obscured by a building representing an invisible road boundary. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building); and
determining the road boundary invisible to the vehicle as the target road boundary(As discussed above, similar to the specification where it discloses the invisible area as being the road boundary obscured by the building, Yasui discloses a region where it determines an unidentified area obscured by an obstacle such as a building. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), and wherein
the determining, based on the real road region and the unknown region, the road boundary invisible to the vehicle(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building), comprises:
converting a collection viewpoint of the real road region and a collection viewpoint of the unknown region into a bird's eye view(Yasui discloses generating an aerial view, similar to bird’s eye view, to determine road area and shading area, which is similar to real road region and unknown region respectively. Yasui’s shading area is similar to the unknown region as it a region obscured by an obstacle such as building. Yasui, Paragraph 48, the image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area), to obtain a converted real road region(Yasui, Paragraph 48, The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner) and a converted unknown region(Yasui, Paragraph 13, the image processing part determines the shading area covered by the object covered by the aerial view data, using the pixel data of the periphery of the shielding area in the overview image data to supplement the determined shielding area);
While Yasui teaches the identification of road lanes and boundary based on the image data, it fails to disclose determining an overlapping region of the converted real road region and the converted unknown region; and
determining a road boundary in the overlapping region as the road boundary invisible to the vehicle.
However, Yang, which is in the same analogous art and that teaches about visual blind area detection discloses determining an overlapping region of the converted real road region and the converted unknown region(Yang discloses determining the intersection(overlapping)region between an extended boundary of a blind spot and road region. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone); and
determining a road boundary in the overlapping region as the road boundary invisible to the vehicle(As discussed above, Yang discloses extended boundary of a blind spot which indicates an invisible region. Yang, paragraph 20, extending each boundary of the blind spot outward by a preset distance, determining a first area based on the extended boundaries, determining the intersection of the first area and the road area, and using the intersection as a warning area. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui with Yang to determine an intersection between blind spot(invisible) region and real road region. By determining the intersection region, it is possible to warn a driver of a danger that could cause an accident.( Yang, paragraph 29, extend each boundary of the visual blind spot by a preset distance, determine a first area based on the extended boundary, and determine the intersection of the first area and the road area, and use the intersection as a warning area).
Regarding claim 21, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the road information comprises at least one of a road surface signal, a lane line, a stop line region, a turning sign, or obstacle information in the road image(Yasui, paragraph 48, The recognition unit 130 includes an image processing unit 132. The image processing unit 132 acquires the image captured by the camera 10, based on the acquired image, generating the aerial view data represented by the way of observing the road area photographed in the image from above in an imaginary manner. The overview data includes a first edge point representing a boundary of a road surface area, such as a road division line, a temporary stop line, and the like, representing a second edge point of a position of a white line drawn on the road surface. Yasui, Paragraph 50, an image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Bu (JP 2022008108 A).
Regarding claim 4, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the detecting, based on the road image, the plurality of road boundaries in the road image comprises:
While the combination of Yasui and Yang teaches the identification of road lanes and boundary based on the image data, it fails to disclose performing semantic segmentation on the road image to obtain a freespace in the road image; and determining the plurality of road boundaries based on a contour line of the freespace.
However, Bu, which is in the same analogous art and that teaches about road area determination device and method discloses performing semantic segmentation on the road image to obtain a freespace in the road image(Bu’s road areas are similar to free space as they are drivable areas for a vehicle. Bu, page 3 line 32, Based on semantic segmentation, the road area is preliminarily (elementarily) extracted from the input image and the reserve road area is determined. Bu, page 7 line 13, the segmentation unit 201 can perform semantic segmentation based on the deep learning network, and may perform semantic segmentation based on, for example, the DeepLab V3 network. When performing semantic segmentation, for interfering substances such as green belts, billboards and utility poles, by considering these interfering objects as the same as roads, the interference caused by these interfering substances is eliminated.); and
determining the plurality of road boundaries based on a contour line of the free space(Bu discloses extracting road contour from outermost contour in the binarized image representing the road boundaries. Bu, page 6 line 20, the contour of the road area can be roughly grasped by searching the outermost contour in the binarized image and ignoring the inner contour included in the outer contour. Bu, page 8 line 26, after removing the relatively small contour, the contour of the road region, that is, the road contour can be obtained ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui and Yang with Bu to identify free space in a road using semantic segmentation. Semantic segmentation provides higher precision in identifying drivable roads as it classifies each pixel in an image captured by a camera. By using semantic segmentation, it is possible to train a machine learning model to recognize roads with different surface and weather conditions with higher accuracy.
Claims 5 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) (hereinafter Yasui) in view of Yang (CN 110456796 A) in further view of Kamiya (US 20220093020 A1).
Regarding claim 5, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone) wherein the determining the target road boundary, which is dangerous to the vehicle, from the plurality of road boundaries comprises at least one of:
determining from the plurality of road boundaries, a road boundary adjacent to a lane where the vehicle is located, as the target road boundary;
determining from the plurality of road boundaries, a road boundary having a distance less than a first preset distance from the vehicle, as the target road boundary; or
determining from the plurality of road boundaries, a road boundary having a road space less than a preset space from the vehicle, as the target road boundary;
The combination of Yasui and Yang fails to disclose determining from the plurality of road boundaries, a road boundary having a distance less than a first preset distance from the vehicle, as the target road boundary.
However, Kamiya, which is in the same analogous art and that teaches about estimating a visible and invisible area of a road surface discloses determining from the plurality of road boundaries, a road boundary having a distance less than a first preset distance from the vehicle, as the target road boundary(Kamiya discloses determining invisible road area( target road boundary) when vehicle is within threshold distance(preset distance). Kamiya, paragraph 87, in S30, a simulation of the display layout is executed, and the visible area Av and the invisible area Ai of the road surface are estimated. Kamiya, paragraph 88, when it is determined that the remaining distance Lr is less than the second threshold value, the process proceeds to S60. Kamiya, paragraph 89, in S60, the in-view angle visible area Av and the in-view angle invisible Ai are estimated again ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui and Yang with Kamiya to determine invisible road area( target road boundary) when vehicle is within threshold distance. By determining invisible area within predetermined distance, it is possible to navigate vehicle early to avoid dangerous road regions.
Regarding claim 9, the combination of Yasui, Yang, and Kamiya teaches the method of claim 5(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone; Kamiya, paragraph 87, in S30, a simulation of the display layout is executed, and the visible area Av and the invisible area Ai of the road surface are estimated.), wherein after the determining the target road boundary, the method further comprises(Similar to the target road boundary that is dangerous based on obstacle information, Yasui discloses image processing to determine a road area that is shielded by an obstacle such as a building which can be a target boundary that is dangerous. Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building):
determining a driving path of the vehicle based on at least one of the target road boundary or the road information(According to the specification, the road information can include a road surface signal, a lane line, or a stop line region. Similarly, Yasui discloses identifying temporary stop line by image processing and generating a travel path. Yasui, paragraph 79, image processing unit 132 using a plurality of images and the second map information 62 obtained from the camera 10 of the time sequence, thereby generating the transformed image obtained in the eye view included in the depth direction, the boundary of the road area of the shielding area road dividing line, temporary stop line and so on more specifically expressed, and the eye view data to the action plan generating part 140 output… according to the second embodiment of the vehicle system 1, it can generate the driving path for driving the vehicle M at a higher precision.); and
controlling driving of the vehicle based on the driving path(Yasui, paragraph 5, the purpose is to provide a vehicle system capable of generating a driving path for driving the vehicle at a higher precision, a control method of the vehicle system and a storage medium.).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Li (CN 111178253 A).
Regarding claim 8, the combination of Yasui and Yang teaches the method of claim 1 (Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein determining the overlapping region of the converted real road region and the converted unknown region comprises(Yang discloses determining the intersection(overlapping)region between an extended boundary of a blind spot and road region. Yang, paragraph 87, extend each boundary of the blind spot by a preset distance, determine the first region based on the extended boundary, determine the intersection of the first region and the road region, and use the intersection as the warning zone):
While the combination of Yasui and Yang teaches about determining and converting a collection viewpoints to bird’s eye view and determination of overlapping region, it fails to disclose fitting a lane line, a stop line region and a turning sign in the converted real road region to obtain first fitting information; fitting a lane line, a stop line region and a turning sign in the converted unknown region to obtain second fitting information.
However, Li, which is in the same analogous art and that teaches about visual perception method and device for automatic driving discloses fitting a lane line, a stop line region and a turning sign in the converted real road region to obtain first fitting information(Li discloses extracting a classification result where it classifies road marker such as stop line and turning mark for fusion and fitting. Li also discloses obtaining a passable area road, which is similar to real road that is drivable, by using fitting mechanism. Li, paragraph 114, an average lane pixel point to each lane in the lane line image curve fitting and calculating the lane line corresponding to the confidence to obtain lane information. Li, paragraph 105, the pavement marker classified network is for network detection image on visual perception of pavement marker, the pavement marker including but not limited to stop line…turning mark. Li, paragraph 26, respectively taking each ellipse fitting the pavement marker image, and calculating the average confidence of each corresponding to the road sign image pixel point, obtaining the road sign information. Li, paragraph 45, rectangular fitting the traffic sign image, and calculating the average confidence of traffic sign image corresponding pixel point to obtain the passable area road condition information. );
fitting a lane line, a stop line region and a turning sign in the converted unknown region to obtain second fitting information(Li discloses determining a region with an obstacle where a vehicle can’t drive. Moreover, it is obvious to one of ordinary skill in the art to determine the region outside the passable road area as being undrivable region similar to the unknown region, where the specification discloses as a region that can be outside a road. Li, paragraph 114, an average lane pixel point to each lane in the lane line image curve fitting and calculating the lane line corresponding to the confidence to obtain lane information. Li, paragraph 105, the pavement marker classified network is for network detection image on visual perception of pavement marker, the pavement marker including but not limited to stop line…turning mark. Li, paragraph 26, respectively taking each ellipse fitting the pavement marker image, and calculating the average confidence of each corresponding to the road sign image pixel point, obtaining the road sign information).
determining, based on the first fitting information and the second fitting information, the overlapping region between the converted real road region and the converted unknown region(After obtaining the first and second fitting information by Li, it is possible to use Yang’s intersection determination unit to determine the intersection between real road region and unknown region(blind spot). Li, paragraph 114, an average lane pixel point to each lane in the lane line image curve fitting and calculating the lane line corresponding to the confidence to obtain lane information. Li, paragraph 105, the pavement marker classified network is for network detection image on visual perception of pavement marker, the pavement marker including but not limited to stop line…turning mark. Li, paragraph 26, respectively taking each ellipse fitting the pavement marker image, and calculating the average confidence of each corresponding to the road sign image pixel point, obtaining the road sign information. Li, paragraph 45 rectangular fitting the traffic sign image, and calculating the average confidence of traffic sign image corresponding pixel point to obtain the passable area road condition information.)
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui and Yang with Li to obtain road fitting information of unknow and drivable road regions based on fitting of lane line and road markers. By Fitting different road lines and signs, it is possible to accurately detect lane markings for the vehicle follow, reducing drifting out of lane. Furthermore, it helps with path navigation, where an autonomous vehicle can be preset with lanes and regions it can enter and regions it has to avoid.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Kamiya (US 20220093020 A1) in further view of Nakano (DE 102019113818 A1).
Regarding claim 10, the combination of Yasui, Yang, and Kamiya teaches method of claim 9(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone; Kamiya, paragraph 87, in S30, a simulation of the display layout is executed, and the visible area Av and the invisible area Ai of the road surface are estimated), wherein the determining the driving path of the vehicle based on the road information comprises(Yasui, paragraph 79, image processing unit 132 using a plurality of images and the second map information 62 obtained from the camera 10 of the time sequence, thereby generating the transformed image obtained in the eye view included in the depth direction, the boundary of the road area of the shielding area road dividing line, temporary stop line and so on more specifically expressed, and the eye view data to the action plan generating part 140 output… according to the second embodiment of the vehicle system 1, it can generate the driving path for driving the vehicle M at a higher precision.):
While the combination of Yasui, Yang, and Kamiya teaches the identification of road lanes and boundary based on the image data, it fails to disclose determining a turning orientation and a turning position of the vehicle based on a road surface signal and a turning sign in the road information; determining the driving path of the vehicle based on the turning orientation and the turning position
However, Nakano, which is in the same analogous art and that teaches about operation control command messages of a vehicle, discloses determining a turning orientation and a turning position of the vehicle based on a road surface signal and a turning sign in the road information( Nakano’s steering of vehicle indicates the changing of turn heading/orientation and turning position of the vehicle. Additionally, Nakano discloses the steering of a vehicle based on road surface markers and signs. Nakano, paragraph 33, a steering assist ECU, which serves to cause the vehicle to follow a more appropriate direction, transmits a control command (i.e., a frame of a steering command) indicating a steering control with an appropriate timing and content, to a CAN bus, based on information obtained by a communication line such as a CAN bus from other ECUs including a sensor ECU, the lane markers on the road surface, or objects and the like in the surrounding area of the vehicle or detected in the direction of travel of the vehicle. As a result of the steering ECU which controls the steering in accordance with the steering control command, the vehicle drives an appropriate route. It should be noted that, as examples of the content of the steering command, the specification of a steering amount by which the vehicle wheels are to be deflected and a steering angle, that is, a turning angle of the vehicle wheels to the right or to the left as a result of the steering operation or as a target value); and
determining the driving path of the vehicle based on the turning orientation and the turning position(Nakano’s steering assist has a capability to keep vehicle within a lane indicating its capability to adjust its orientation dynamically based on a target path, such as based on surface lane. Moreover, as discusses above, the steering control can be implemented based on the surface marks, and determination of target path indicates its steering control. Nakano, paragraph 171, the state data indicates a state that a lane keeping function that is one of the steering assist functions in the vehicle 20A act, is activated (on), and the vehicle 20A located near the left side within the lane in which the vehicle is 20A Is on the way. That is, the above-mentioned state is such a state that, by the control by means of the lane keeping function as a target path).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui, Yang, and Kamiya with Nakano to determine steering angle and position of a vehicle based on the road surface mark and sign. By determining the position and orientation of a vehicle based on the turning lane markers and signs, it is possible for a vehicle to automatically navigate its path without human intervention.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Kamiya (US 20220093020 A1) in further view of Tao (US 20210027629 A1).
Regarding claim 11, the combination of Yasui, Yang, and Kamiya teaches the method of claim 9(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the controlling the driving of the vehicle based on the driving path comprises(Yasui, paragraph 79, image processing unit 132 using a plurality of images and the second map information 62 obtained from the camera 10 of the time sequence, thereby generating the transformed image obtained in the eye view included in the depth direction, the boundary of the road area of the shielding area road dividing line, temporary stop line and so on more specifically expressed, and the eye view data to the action plan generating part 140 output… according to the second embodiment of the vehicle system 1, it can generate the driving path for driving the vehicle M at a higher precision.):
While the combination of Yasui, Yang, and Kamiya teaches the identification of road lanes and boundary based on the image data, it fails to disclose updating the driving path based on obstacle information in the road information, to obtain an updated path; and controlling the driving of the vehicle based on the updated path.
However, Tao, which is in the same analogous art and that teaches about blind area processing for planning and control of an autonomous vehicle, teaches updating the driving path based on obstacle information in the road information, to obtain an updated path(Tao, paragraph 64, the navigation system may determine a series of speeds and directional headings to affect movement of the autonomous driving vehicle along a path that substantially avoids perceived obstacles while generally advancing the autonomous driving vehicle along a roadway-based path leading to an ultimate destination….the navigation system may update the driving path dynamically while the autonomous driving vehicle is in operation); and
controlling the driving of the vehicle based on the updated path(Tao, paragraph 64, the navigation system may update the driving path dynamically while the autonomous driving vehicle is in operation).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui, Yang, and Kamiya with Tao to update the driving path of a vehicle based on detected obstacle. By updating travel route(path) based on detected obstacle, it is possible for a vehicle to safely maneuver away from a hazard on the road, even if the hazard is not stored in the map data.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Kamiya (US 20220093020 A1) in further view of Zeng (CN 113587951 A).
Regarding claim 12, the combination of Yasui, Yang, and Kamiya teaches the method of claim 9(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the determining the driving path of the vehicle based on the target road boundary comprises(Yasui, paragraph 79, image processing unit 132 using a plurality of images and the second map information 62 obtained from the camera 10 of the time sequence, thereby generating the transformed image obtained in the eye view included in the depth direction, the boundary of the road area of the shielding area road dividing line, temporary stop line and so on more specifically expressed, and the eye view data to the action plan generating part 140 output… according to the second embodiment of the vehicle system 1, it can generate the driving path for driving the vehicle M at a higher precision.):
While the combination of Yasui, Yang, and Kamiya teaches the identification of road lanes and boundary based on the image data, it fails to disclose updating map data of a position of the vehicle based on the target road boundary, to obtain an updated map; and determining the driving path of the vehicle based on the updated map.
However, Zeng, which is in the same analogous art and that teaches about a map information updating system, discloses updating map data of a position of the vehicle based on the target road boundary, to obtain an updated map(As discussed above, a target road boundary can be an area shielded by an obstacle. Zeng discloses a road area obstructed by an obstacle and updating a map based on the obstacle. Zeng, paragraph 67, determining the obstacle sensing information corresponding to the obstacle position; marking the obstacle position corresponding to the obstacle sensing information in the currently used map, and determining the marked map as the updated map ); and determining the driving path of the vehicle based on the updated map(Zeng, paragraph 65, plan the driving path according to the obstacle perception information and obstacle type to obtain the latest driving path).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui, Yang, and Kamiya with Zeng to update map data based on the detection of target boundary such as boundary obscured by an obstacle and unidentified regions. By updating the map data, it is possible for the vehicle to modify its travel path to avoid dangers that could cause an accident. Furthermore, updated maps can be transmitted to nearby vehicles to modify their own planned routes.
Claims 13, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Cheon (CN 113147747 A).
Regarding claim 13, the combination of Yasui and Yang teaches the method of claim 1 (Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein after the determining the target road boundary, the method further comprises:
The combination of Yasui and Yang fails to disclose controlling the vehicle based on a relationship between the target road boundary and a driving state of the vehicle.
However, Cheon, which is in the same analogous art and that teaches about vehicle assisting device and method discloses controlling the vehicle based on a relationship between the target road boundary and a driving state of the vehicle(According to the specification, one of the cases of relationship between the target road boundary and a driving state of the vehicle is a case where the target road boundary is connected with a lane of an ego vehicle. Similarly, Cheon discloses determining a target road boundary (a boundary obscured by an obstacle ob3) located inside the driving lane of an ego vehicle that can touch the vehicle if the vehicle continues to drives in the same path. After determination of the obstacle, Cheon proceeds to brake the vehicle. Furthermore, as demonstrated by Cheon in figure 11, a target road boundary (a boundary obscured by an obstacle ob3) is inside the driving lane of the ego vehicle, where the vehicle can touch the obstacle (ob3), if it continues to drives in the same path. Cheon, paragraph 150, the controller 140 can determine the similarity between the first obstacle ob1 and the third obstacle ob3 is equal to or greater than the first threshold value; and the controller 140 may perform collision avoidance control based on the collision path stored in the storage device 150 and/or the memory 142 in case the vehicle 1 is fully braked ).
PNG
media_image1.png
651
512
media_image1.png
Greyscale
Cheon Figure 11: A target boundary (a boundary obscured by an obstacle ob3) connected to the lane of the ego vehicle
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui and Yang with Cheon to control a vehicle after determining if a target boundary (a boundary obscured by an obstacle) is connected to a lane the vehicle is located on. By controlling a vehicle when a region of obstacle is within the lane of a vehicle, it is possible to prevent a collision by reacting either by braking or driving away from the region. Early determining of an obstacle region from a distance helps easily maneuver the vehicle avoid collision, instead of trying to control the vehicle right before reaching the obstacle.
Regarding claim 14, the combination of Yasui, Yang, and Cheon teaches the method of claim 13(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the relationship between the target road boundary and the driving state of the vehicle comprises at least one of:
a case where a distance between the overlapping region where the target road boundary is located and a road intersection ahead of the vehicle is less than a second preset distance;
a case where a distance between the overlapping region and a position of the vehicle is less than a third preset distance;
a case where an included angle between a driving direction of the vehicle and the target road boundary is less than a preset angle; or
a case where the target road boundary is connected with a lane where the vehicle is located(As discussed above, the specification discloses the target road boundary is connected with the lane where the vehicle is located if a vehicle will touch the target road boundary by continuing to drive in the lane. Similarly, as demonstrated by Cheon in figure 11, a target road boundary (a boundary obscured by an obstacle ob3) is inside the driving lane of an ego vehicle that can touch the vehicle if the continues to drives in the same path.).
PNG
media_image1.png
651
512
media_image1.png
Greyscale
Cheon Figure 11: A target boundary (a boundary obscured by an obstacle ob3) connected to the lane of the ego vehicle
Regarding claim 15, the combination of Yasui, Yang, and Cheon teaches the method of claim 13(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein the controlling the vehicle comprises: controlling the vehicle to enter a brake state from the driving state(Cheon discloses after determining an obstacle is within the lane of the ego vehicle, it can perform collision avoidance by braking the vehicle. Cheon, FIG. 11 shows that the male pedestrian completely exceeds the second obstacle ob2 without detecting the presence of the vehicle 1 and does not reduce the transverse speed of the state. Cheon, paragraph 150, the controller 140 can determine the similarity between the first obstacle ob1 and the third obstacle ob3 is equal to or greater than the first threshold value; and the controller 140 may perform collision avoidance control based on the collision path stored in the storage device 150 and/or the memory 142 in case the vehicle 1 is fully braked), or controlling the vehicle to drive away from the target road boundary( Yasui, Paragraph 54, The steering control section 166 controls the steering apparatus 220 according to the bending condition of the target track stored in the memory.).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Kamiya (US 20220093020 A1) in further view of Tao (US 20210027629 A1) in further view of Lee (US 20190042860 A1).
Regarding claim 16, the combination of Yasui and Yang teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein after the determining the target road boundary, the method(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building) further comprise sat least one of:
setting a region of interest based on the target road boundary and (As discussed above, target road boundary can be an area where it’s obscured by an obstacle. Tao discloses defining a region of interest based on proximity to a target path while ignoring target road boundaries such as blind areas and obscuring objects. Tao, paragraph 31, the ADV can ignore blind areas and objects that are outside of a region of interest 212. The region of interest can be defined based on proximity to a target path 213 of the ADV ),
While the combination of Yasui, Yang, Kamiya, and Tao teaches determining region of interest based on target road boundaries such as blind spot and shielded boundary, it specifically fails to disclose obtaining an image corresponding to the region of interest based on a first resolution, wherein the road image is obtained according to a second resolution, and the second resolution is less than the first resolution; or obtaining the image corresponding to the region of interest based on a first frame rate, wherein the road image is obtained based on a second frame rate, and the second frame rate is less than the first frame rate.
However, Lee, which is in the same analogous art and that teaches about a method and apparatus of detecting an object of interest discloses obtaining an image corresponding to the region of interest based on a first resolution(Lee, paragraph 6, acquiring an input image, setting a region of interest (ROI) in the input image, generating a restoration image corresponding to the ROI, a resolution of the restoration image being greater than a resolution of the input image), wherein the road image is obtained according to a second resolution, and the second resolution is less than the first resolution(The input image is similar to the second resolution as it has lower resolution than the first resolution. Additionally, the input image may include road vanishing point or road markings representing a road image. Lee, paragraph 54, The ROI includes at least one of a portion of a region of the input image or at least one object included in the input image. An object may include, for example, a vehicle other than vehicle that is being driven, a road vanishing point, a road marking, a pedestrian, a vehicle, a traffic light, a lane marking, a traffic sign, a human, an animal, a plant, and a building, but the object is not limited thereto.);
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui, Yang, Kamiya, and Tao with Lee to obtain an image of region of interest based different resolutions. By collecting images of region of interest with different quality of resolution, it is possible to assign higher resolutions for critical areas that are helpful for safe navigation such as images of obstacles, traffic signs and lane lines while less critical areas can be obtained with lesser resolution.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Yasui (CN 111717201 A) in view of Yang (CN 110456796 A) in further view of Cheon (CN 113147747 A) in further view of Sakagami (JP 2006285693 A).
Regarding claim 17, the combination of Yasui, Yang, and Cheon teaches the method of claim 1(Yasui, paragraph 48, the recognition unit 130 includes an image processing unit 132; Yang, paragraph 87, determine the intersection of the first region and the road region, and use the intersection as the warning zone), wherein after the determining the target road boundary, the method(Yasui, Paragraph 50, An image processing unit 132 determines a road surface area included in the generated overview image data, such as a shielding area (so called occlusion area) shielded by an object such as a building) further comprises:
collecting road environment information around the target road boundary(The specification discloses one of road environment information is an obstacle information of a road boundary. Cheon discloses determining obstacle information of an area of the vehicle, which is similar to a target boundary obscured by an obstacle. Cheon, paragraph 18, generating a collision avoidance path for avoiding the collision between the vehicle and the first obstacle; and storing the generated collision path and information of the first obstacle. Cheon, paragraph 107, The plurality of receiving channels here can receive the wireless electric wave reflected by the obstacle in each area, each area is divided into a predetermined angle with the center of the vehicle 1 front part as the central point );
generating notification information based on the road environment information(Cheon discloses comparing different obstacles to determine their similarity, it performs the comparison by determining if the similarity is greater than a preset threshold value. After the determining the similarity, the notification unit sends a control signal for generating a warning signal. The determining and comparison of similarity indicates Cheon’s capability to determine obstacle information such as its’s length and position. Based on Cheon’s notification, it would be obvious to one of ordinary skill in the art to incorporate the generating of notification regarding the obstacle information present around the area of the vehicle. Cheon, paragraph 27, when the similarity between the first obstacle ob1 and the third obstacle ob3 is greater than the third threshold value, the controller 140 can according to the stored avoidance path radial notification unit 160 sends a control signal for generating a warning signal); and
While the combination of Yasui, Yang, and Cheon disclose generating notification regarding target road boundary such as boundary shielded by obstacle it fails to disclose sending the notification information to a rear vehicle of the vehicle, wherein the rear vehicle and the vehicle are located in a same lane and drive in a same direction.
However, Sakagami, which is in the same analogous art and that teaches about an inter-vehicle communication system, discloses sending the notification information to a rear vehicle of the vehicle, wherein the rear vehicle and the vehicle are located in a same lane and drive in a same direction(Sakagami, page 11 line 22, the other moving body data notifying unit 59 notifies the object data (other moving body data) to the other vehicle C that has not detected the object among the information sharing vehicles determined by the information sharing vehicle determining unit 57).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yasui, Yang, and Cheon with Sakagami to notify the rear vehicle regarding a road obstruction caused by an obstacle which is a target road boundary. By sending notification to nearby vehicle, it is possible to warn other vehicles of unknown region where an obstacle maybe present, which can prevent an accident.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BESUFEKAD LEMMA TESSEMA whose telephone number is (571)272-6850. The examiner can normally be reached Monday - Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at 5712727298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BESUFEKAD LEMMA TESSEMA/Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights