Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 7, 11, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Okabe et al (US20230344955).
Regarding Claim 1. Okabe teaches An image generation apparatus comprising
a processor that acquires a first image in which a surrounding environment of a vehicle imaged by an imaging apparatus provided in the vehicle is shown (Okabe, abstract, the invention describes a display control apparatus including: an image inputter; a vehicle periphery image generator that generates a vehicle periphery image; a subject image outputter that outputs a subject image; and an image superimposer that generates a display image by superimposing the subject image on the vehicle periphery image. An azimuth of a first line of sight is substantially identical to an azimuth of a second line of sight.
[0042] Image inputter 110 receives images, which have been obtained by capturing the periphery of vehicle 1, from four capturers 10 and outputs captured images that have been subjected to distortion correction such as lens distortion correction.),
generates, based on the first image, a plurality of second images corresponding to horizontal components viewed from a virtual viewpoint provided above the vehicle in a plurality of line-of-sight directions that are shifted in a direction of rotation around the vehicle and are different from each other (Okabe, [0044] Vehicle periphery image generator 130 generates, based on the image received by image inputter 110, a vehicle periphery image in which the periphery of vehicle 1 is viewed from a first virtual visual point. Specifically, vehicle periphery image generator 130 projects the respective captured images of four capturers 10 on a projection plane (for example, a flat plane corresponding to the road surface) to combine the respective captured images into one overhead image.
[0047] Since projection images can be generated based on captured images of each capturer 10, respectively, a panoramic projection image of vehicle 1 can be obtained by four capturers 10 as illustrated in FIG. 4, for example. Pl in FIG. 4 is a projection plane by capturer 10 in the right portion of vehicle 1. P2 in FIG. 4 is a projection plane by capturer 10 in the front portion of vehicle 1. P3 in FIG. 4 is a projection plane by capturer 10 in the left portion of vehicle 1. P4 in FIG. 4 is a projection plane by capturer 10 in the rear portions of vehicle 1.
[0048] An image in which the projection image described above is viewed from a virtual visual point positioned above vehicle 1 is an overhead image. That is, vehicle periphery image generator 130 generates an overhead image in which a projection image is viewed from a first virtual visual point.
Although Okabe didn’t explicitly teach “a plurality of line-of-sight directions”, the projection images are created at a above viewpoint pointing to different directions around the vehicle. Therefore, it is obvious to a person with ordinary skill in the art that those projection images are views from different line-of-sight directions around the vehicle.), and switches and outputs the plurality of second images to a display device (Okabe, [0049] One image into which four overhead images based on the respective captured images of capturers 10 provided at the front, rear, left, and right of vehicle 1, respectively, are combined is a panoramic image. That is, vehicle periphery image generator 130 generates a panoramic image as a vehicle periphery image. A change of a visual point position, from which an image is viewed, from the actual visual point (the position of capturer 10) to a virtual visual point is also referred to as visual point conversion. Accordingly, subject image outputter 140 and image superimposer 150 generate a display image obtained by superimposing a subject image, in which a three-D (three-dimensional) model of vehicle 1 is viewed from a virtual visual point (second virtual visual point V2), on a blind-spot region in a vehicle periphery image.
Therefore, the plurality of projection images are stitched together as a panoramic image as a vehicle periphery image and displayed on the screen.).
Regarding Claim 7. Okabe further teaches The image generation apparatus according to claim 1, wherein the processor
generates a vehicle model image based on a three-dimensional vehicle model corresponding to the vehicle, displays the vehicle model image to be superimposed on each of the plurality of second images (Okabe, [0054] However, since vehicle 1 is not reflected in capturer 10, it is necessary to complement the figure of vehicle 1. Further, within the range of the second projection, at least a range positioned on a lower side of vehicle 1 is a blind-spot region that is not reflected in capturer 10, and becomes a blank region in a vehicle periphery image. Accordingly, subject image outputter 140 and image superimposer 150 generate a display image obtained by superimposing a subject image, in which a three-D (three-dimensional) model of vehicle 1 is viewed from a virtual visual point (second virtual visual point V2), on a blind-spot region in a vehicle periphery image.), and
restricts a height of the vehicle model when generating the vehicle model image (Okabe, [0059] Further, as illustrated in FIG. 6B, as visual point position VV is lowered more to the rear of vehicle 1, the second projection is performed at the shallower angle, and thus, among figures on projection plane PP, a figure projected on the dashed-line portion on the side of visual point position VV of vehicle 1 is displayed, compressed (collapsed) in the up-down direction, on the display screen of display 30. Accordingly, it becomes impossible to understand what is displayed on display 30.).
Claim 11 is similar in scope as Claim 1, and thus is rejected under same rationale.
Claim 17 is similar in scope as Claim 7, and thus is rejected under same rationale.
Claims 2-6, 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Okabe et al (US20230344955) in view of Murasumi et al (US20160182823).
Regarding Claim 2. Okabe fails to explicitly teach, however, Murasumi teaches The image generation apparatus according to claim 1, wherein the processor sets depression angles of the plurality of line-of-sight directions such that a part of a vehicle model indicating the vehicle is shown in the plurality of second images (Murasumi, abstract, the invention describes an image generation unit uses a plurality of captured image data and vehicle body data of a vehicle to continuously generate a virtual perspective image indicating the vehicle body of the vehicle and the periphery of the vehicle as seen from a virtual perspective (VP) positioned within the cabin of the vehicle. An image control unit alters the angle in a plan view of the line of sight of the VP in a manner such that the line of sight circles the surroundings of the vehicle, and moves the position of the VP in the front-back direction of the vehicle.
[0024] In the image generation device of the first aspect, when at least one of the plurality of cameras has a depression angle of an optical axis different from the other cameras, the generation unit may be configured to project the plurality of captured image data to a virtual projection plane and to generate the virtual perspective image by using an area of a part of the projection plane. Also, the control unit may be configured to adjust a depression angle of the line of sight so that a non-projection area, which is positioned outside a projection area of the projection plane to which the plurality of captured image data is to be projected, is not included in the virtual perspective image.
[0037] According to the above configuration, since the position of the virtual perspective is moved in the front-back direction of the vehicle, it is possible to reduce parts of the vehicle body to be indicated in the virtual perspective image and to improve the visibility of a photographic subject image of the periphery of the vehicle to be included in an in-vehicle perspective image.
Okabe, [0035] The plurality of capturers 10 is attached obliquely downward with an angel of depression so as to capture mainly the road surface (see also FIG. 3). The visual-field angle of each capturer 10 is equal to or greater than 190 degrees, and four capturers 10 allow the entire surroundings of vehicle 1 to come into sight. The plurality of capturers 10 transmits captured images in the four directions to display control apparatus 100.).
Okabe and Murasumi are analogous art because they both teach method of generating periphery image for vehicle. Murasumi further teaches the image capturers has a depression angel to include part of the vehicle body. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the vehicle periphery image generation method (taught in Okabe) to further adjust the image capturers with a depression angel to capture part of vehicle body (taught in Murasumi), so as to continuously generate an in-vehicle perspective image while altering a direction of a line of sight of a virtual perspective and to indicate a shape of a periphery of a vehicle as seen from a driver's perspective so as to circle surroundings of the vehicle (Murasumi, [0002-0005]).
Regarding Claim 3. The combination of Okabe and Murasumi further teaches The image generation apparatus according to claim 1, wherein the processor sets depression angles of the plurality of line-of-sight directions so as to smoothly change with respect to a direction of the horizontal components of the plurality of line-of-sight directions (Murasumi, [0030] In the image generation device of the first aspect, if the position of the virtual perspective is a position except for a specific position, the control unit may keep the depression angle of the line of sight of the virtual perspective at a specific angle at which the virtual perspective image does not include the non-projection area when the line of sight is made to face towards a direction in which the projection area becomes smallest on the projection plane, and if the position of the virtual perspective is the specific position, the control unit may make the depression angle of the line of sight of the virtual perspective smaller than the specific angle.
[0031] According to the above configuration, the virtual perspective image can indicate a photographic subject image relatively distant from the vehicle with respect to a specific direction of the vehicle while simplifying the movement of the line of sight of the virtual perspective.
[0035] According to the above configuration, since the position of the virtual perspective is moved in the front-back direction of the vehicle, it is possible to reduce parts of the vehicle body to be indicated in the virtual perspective image and to improve the visibility of a photographic subject image of the periphery of the vehicle to be included in an in-vehicle perspective image.).
The reasoning for combination of Okabe and Murasumi is the same as described in Claim 2.
Regarding Claim 4. The combination of Okabe and Murasumi further teaches The image generation apparatus according to claim 2, wherein the processor sets the depression angles of the plurality of line-of-sight directions, based on a square measure of an area in which the vehicle model is shown in the plurality of second images (Murasumi, [0028] In the image generation device of the first aspect, the control unit may be configured to alter the depression angle of the line of sight, in correspondence to a size of the projection area of the projection plane in a direction towards which the line of sight is made to face.
[0029] According to the above configuration, since the depression angle of the line of sight is altered in correspondence to the size of the projection area of the projection plane, the virtual perspective image can indicate a photographic subject image relatively distant from the vehicle.).
The reasoning for combination of Okabe and Murasumi is the same as described in Claim 2.
Regarding Claim 5. The combination of Okabe and Murasumi further teaches The image generation apparatus according to claim 2, wherein the processor sets the depression angles of the plurality of line-of-sight directions, based on a dimension of an area in which the vehicle model is shown in the plurality of second images (Murasumi, [0240] As described above, on the projection plane TS, the size (a length in the direction of getting away from the vehicle 9) of the projection area R1 becomes smallest at the left and the right of the vehicle 9. Therefore, the size (a length in the direction of getting away from the vehicle 9) of the projection area R1 is larger at the front or rear of the vehicle 9 on the projection plane TS, as compared to the left and the right of the vehicle 9. For this reason, when the line of sight of the virtual perspective VP is made to face towards the front or rear of the vehicle 9, the in-vehicle perspective image does not include the non-projection areas R2 even though the line of sight of the virtual perspective VP is made to face upwards. In the fifth illustrative embodiment, the image control unit 20a is configured to alter the depression angle of the line of sight of the virtual perspective VP within a range in which the invehicle perspective image does not include the non-projection areas R2, in correspondence to the size of the projection area R1.).
The reasoning for combination of Okabe and Murasumi is the same as described in Claim 2.
Regarding Claim 6. The combination of Okabe and Murasumi further teaches The image generation apparatus according to claim 1, wherein the processor sets a location of the virtual viewpoint for each of the plurality of line-of-sight directions such that a part of a vehicle model indicating the vehicle is shown in the plurality of second images (Murasumi, [0037] According to the above configuration, since the position of the virtual perspective is moved in the front-back direction of the vehicle, it is possible to reduce parts of the vehicle body to be indicated in the virtual perspective image and to improve the visibility of a photographic subject image of the periphery of the vehicle to be included in an in-vehicle perspective image.
[0169] FIG. 8 depicts a moving path of the virtual perspective VP in the circling mode M2, showing the vehicle 9 in a plan view. In FIG. 8, each solid arrow indicates the virtual perspective VP, and a start point of the solid arrow indicates a position of the virtual perspective VP. A direction of the solid arrow indicates a direction of the line of sight of the virtual perspective VP in a plan view. That is, an angle of the solid arrow relative to the front-back direction of the vehicle 9 corresponds to a plan view angle of the line of sight of the virtual perspective VP.).
The reasoning for combination of Okabe and Murasumi is the same as described in Claim 2.
Claim 12 is similar in scope as Claim 2, and thus is rejected under same rationale.
Claim 13 is similar in scope as Claim 3, and thus is rejected under same rationale.
Claim 14 is similar in scope as Claim 4, and thus is rejected under same rationale.
Claim 15 is similar in scope as Claim 5, and thus is rejected under same rationale.
Claim 16 is similar in scope as Claim 6, and thus is rejected under same rationale.
Allowable Subject Matter
Claims 8-10, 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding Claim 8, it recites “The image generation apparatus according to claim 7, wherein the processor calculates a color value of the vehicle model by executing shading processing on the vehicle model before height restriction, and applies the color value to the vehicle model after the height restriction” in the context of Claim 8.
The prior arts of record either alone or in combination fails to teach or suggest the above quoted limitation of Claim 8. Therefore, Claim 8 is allowable over prior art.
Regarding Claim 9, it recites “The image generation apparatus according to claim 7, wherein the processor compresses a body above a wheel with respect to the vehicle model to restrict the height of the vehicle model” in the context of Claim 9.
The prior arts of record either alone or in combination fails to teach or suggest the above quoted limitation of Claim 9. Therefore, Claim 9 is allowable over prior art.
Regarding Claim 10, it recites “The image generation apparatus according to claim 7, wherein the processor restricts the height of the vehicle model by moving a polygon vertex of the vehicle model along a straight line extending from the virtual viewpoint” in the context of Claim 10.
The prior arts of record either alone or in combination fails to teach or suggest the above quoted limitation of Claim 10. Therefore, Claim 10 is allowable over prior art.
Claim 18 recites similar limitations as discussed above with regard to claim 8. Therefore, claim 18 is allowable over prior art.
Claim 19 recites similar limitations as discussed above with regard to claim 9. Therefore, claim 19 is allowable over prior art.
Claim 20 recites similar limitations as discussed above with regard to claim 10. Therefore, claim 20 is allowable over prior art.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Fujii et al (US20160065944), the invention describes an image display apparatus for converting an image into a virtual viewpoint image so that the image of an object is not distorted.
[0023] … The exemplary embodiment relates to an image display system for converting an image of a camera installed to capture images of vehicles traveling on a roadway into a virtual viewpoint image and displaying the virtual viewpoint image. The image display system according to the exemplary embodiment extracts a region where an object exists (hereinafter referred to as "object region") from images captured in an area where monitoring cameras are installed at four sides, i.e., front, back, right, and left (hereinafter referred to as "object three-dimensional model generation area"), and generates a three-dimensional model from an object region (hereinafter referred to as "object three-dimensional model").
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished appications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/Primary Examiner, Art Unit 2619
Prosecution Insights