DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-12 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 2/2/24, 3/4/25, and 5/20/25 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “information processing device for generating”, “detection data obtainer that obtains”, “data complementation that extracts”, and “data synthesizer that synthesizes” in claim 1, “data complementation adds” in claim 3, “data complementation extracts” in claim 4, “data synthesizer generates” in claim 5, “information processing device generates” in claim 6, “detection data obtainer obtains”, “data complementation extracts”, and “data synthesizer generates” in claim 7, “image generator that generates” and “image generator generates” in claim 8, and “information processing device for generating”, “detection data obtainer that obtains” and “synthesizer that extracts” in claim 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 2 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites the limitation "the three-dimensional data" in line 5. It is unclear and indefinite which “three-dimensional data” is being referred to, as there are multiple instances of “three-dimensional data” previously recited.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 7-12 are rejected under 35 U.S.C. 103 as being unpatentable over Hiroyoshi (JP 2013025528 A, see provided machine translation).
Regarding claim 1, Hiroyoshi teaches, An information processing device for generating driving assistance information for a vehicle, the information processing device comprising (Fig. 1: image generation device 100; Abstract: “An image generation device for vehicles comprises: a blind spot information estimating part 25 for estimating information including the shape and the color of another vehicle in blind spot parts on the basis of image information on visible image parts of the another vehicle included in an image around an own vehicle having a three dimensional structure; Para. 0021: “Referring back to FIG. 1, the ROM 12 stores a dedicated program for realizing a vehicle periphery image generation function, which will be described later, and various kinds of information necessary for executing the program. In response to a request from the CPU 10, the ROM 12 reads and inputs the stored information to the CPU 10”; Paras. 0034-0035: another vehicle is detected and traveling state is determined):
a detection data obtainer that obtains detection data, the detection data being three-dimensional data obtained by detecting an object located outside the vehicle (Para. 0005: “a vehicle periphery image having a three dimensional structure is generated based on a plurality of images obtained by photographing the periphery of a vehicle by a plurality of cameras mounted on the vehicle”; Para. 0010; Para. 0011: “Here, the vehicular image generating apparatus 100 of the present embodiment generates a three dimensionally structured image of the vehicular periphery by using the image obtained by photographing the region of the vehicular periphery by the imaging devices 11A to 11D… In order to generate a vehicle periphery image having a three dimensional structure, it is necessary to detect three dimensional coordinate information (X, Y, Z) of a subject (object) present in an imaging region in addition to an image captured by a camera”);
a data complementation that extracts object data from a plurality of pieces of three-dimensional data of objects stored in advance, the object data being three-dimensional data of the object corresponding to the detection data, and complements the detection data with the object data to generate complementary data that is three-dimensional data (Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data”; Para. 0033: “The vehicle-periphery image generating unit 23 generates a vehicle-periphery image having a three dimensional structure… Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13 in association with the photographing time information of the photographed image used for the generation, and inputs the vehicle-periphery image to the vehicle-image detecting unit 24”; Para. 0034: there is template matching in which the vehicle detection unit 24 cuts out an image portion of the detected other vehicle included in the vehicle periphery image data and inputs it to the blind spot information estimation unit 25; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Paras. 0061-0066: image complementing process);
and a data synthesizer that synthesizes the complementary data and vehicle data, the vehicle data being three-dimensional data of the vehicle, to generate synthesized data that is three-dimensional data including the vehicle and an external area of the vehicle (Abstract; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13 and there is a visible side surface image; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD 16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0043: the image complementing unit stores the complemented vehicle periphery image; Paras. 0061-0066: image complementing process; Para. 0065: “the image complementing unit 26 generates a three dimensional image of the retrieved vehicles”; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image as the synthesized data).
Hiroyoshi discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for obtaining detection data, complementing the detection data with extracted object data to generate complementary data, and synthesizing the complementary data and vehicle data to generate synthesized data since modifications, improvements, equivalents, and the like can be made to the present invention (Hiroyoshi, Para. 0115) and in order to more accurately generate an image of another vehicle (Hiroyoshi, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 1.
Regarding claim 2, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), wherein the detection data is three-dimensional data of a part of an outer shape of the object located outside the vehicle (Para. 0032: “The projection-image generating unit 22 generates a projection-image dataset corresponding to each captured-image dataset based on the captured-image datasets of the imaging devices RAM 13 to 111 D stored in 111A and the three dimensional coordinate information corresponding to each captured-image dataset input from the coordinate-information detecting unit 21. Specifically, the projection image generation unit 22 recognizes the shape of the object included in each piece of captured image data from the three dimensional coordinate information corresponding to each piece of captured image data”; Para. 0005: “a vehicle periphery image having a three dimensional structure is generated based on a plurality of images obtained by photographing the periphery of a vehicle by a plurality of cameras mounted on the vehicle”; Para. 0010; Para. 0011: “Here, the vehicular image generating apparatus 100 of the present embodiment generates a three dimensionally structured image of the vehicular periphery by using the image obtained by photographing the region of the vehicular periphery by the imaging devices 11A to 11D… In order to generate a vehicle periphery image having a three dimensional structure, it is necessary to detect three dimensional coordinate information (X, Y, Z) of a subject (object) present in an imaging region in addition to an image captured by a camera”),
and the object data includes three-dimensional data of an entirety of the outer shape of the object, the three-dimensional data being stored in advance (Para. 0023: “The HDD (Hard disk drive) 16 stores a vehicular image database 300 in which whole images (3D models) having three dimensional structures of a plurality of types of vehicles are stored”; Para. 0077: “information indicating a correspondence relationship between the shape of the side surface of the car and the shape of the whole car is stored in the ROM 12, and the shape is estimated by comparing the information with the shape of the visible side surface image of the other car 2”; Para. 0033: “The vehicle-periphery image generating unit 23 generates a vehicle-periphery image having a three dimensional structure… Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13 in association with the photographing time information of the photographed image used for the generation, and inputs the vehicle-periphery image to the vehicle-image detecting unit 24”; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD 16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”).
Regarding claim 3, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), wherein the data complementation adds a portion of the object data, the portion excluding a portion overlapping with the detection data, to the detection data to generate the complementary data (Abstract; Para. 0033; Para. 0034: there is template matching in which the vehicle detection unit 24 cuts out an image portion of the detected other vehicle included in the vehicle periphery image data and inputs it to the blind spot information estimation unit 25; Para. 0036: blind spot information of another vehicle is estimated based on the image information of the visible portion data in which the blind spot of other vehicles is outside the imaging range of the imaging devices; Para. 0042: determining the unestimatable part of the blind spot using the stored three dimensional CG model; Paras. 0061-0066: image complementing process; Note: the Examiner interprets the blind spot information being outside the imaging range of the imaging devices, and determining the unestimatable part of the blind spot using the stored three dimensional CG model as data that does not overlap with the detection data).
Regarding claim 4, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), further comprising: a storage that stores the plurality of pieces of three-dimensional data of the objects (Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD 16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”),
wherein the data complementation extracts, from the plurality of pieces of three-dimensional data of the objects stored in the storage, data having a high correlation with the detection data as the object data (Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD 16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0094; Para. 0107: “When there is a three dimensional CG model of a vehicle type having an image part matching a visible image part among three dimensional CG models of a plurality of vehicle types stored in a vehicle image database 300, an image of a blind spot of another vehicle is complemented by using a complementary image generated by using image data of the three dimensional CG model.; Note: the Examiner interprets a three dimensional CG model being of the same car type/matching a visible part as a high correlation with the detection data).
Regarding claim 5, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), further comprising: a storage that stores the vehicle data (Paras. 0021-0023: ROM 12, RAM 13, HDD 16 are storages; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13 and there is a visible side surface image; Para. 0042),
wherein the data synthesizer generates the synthesized data based on the vehicle data stored in the storage (Abstract; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13, there is a visible side surface image, and image complementing is performed; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0043: the image complementing unit stores the complemented vehicle periphery image; Paras. 0061-0066: image complementing process; Para. 0065: “the image complementing unit 26 generates a three dimensional image of the retrieved vehicles”; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image as the synthesized data).
Regarding claim 7, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), wherein the detection data obtainer obtains a plurality of pieces of the detection data that are pieces of detection data of objects located at least behind and at a side of the vehicle (As shown in Fig. 2, imaging device 11D is positioned at the back/rear of host vehicle 1 and imaging devices 11B and 11C are positioned on the sides of host vehicle 1; As shown in Fig. 16, another vehicle 2 is in back of and to the side of host vehicle 1; Para. 0113; Para. 0005: “a vehicle periphery image having a three dimensional structure is generated based on a plurality of images obtained by photographing the periphery of a vehicle by a plurality of cameras mounted on the vehicle”; Para. 0010; Para. 0011: “Here, the vehicular image generating apparatus 100 of the present embodiment generates a three dimensionally structured image of the vehicular periphery by using the image obtained by photographing the region of the vehicular periphery by the imaging devices 11A to 11D… In order to generate a vehicle periphery image having a three dimensional structure, it is necessary to detect three dimensional coordinate information (X, Y, Z) of a subject (object) present in an imaging region in addition to an image captured by a camera”),
the data complementation extracts a plurality of pieces of the object data based on the plurality of pieces of the detection data, and complements the plurality of pieces of the detection data with the plurality of pieces of the object data to generate a plurality of pieces of the complementary data (Paras. 0113-0114; As shown in Fig. 2, imaging device 11D is positioned at the back/rear of the vehicle and imaging devices 11B and 11C are positioned on the sides of the vehicle; As shown in Fig. 16, another vehicle 2 is in back of and to the side of host vehicle 1; Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data”; Para. 0033: “The vehicle-periphery image generating unit 23 generates a vehicle-periphery image having a three dimensional structure… Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13 in association with the photographing time information of the photographed image used for the generation, and inputs the vehicle-periphery image to the vehicle-image detecting unit 24”; Para. 0034: there is template matching in which the vehicle detection unit 24 cuts out an image portion of the detected other vehicle included in the vehicle periphery image data and inputs it to the blind spot information estimation unit 25; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Paras. 0061-0066: image complementing process),
and the data synthesizer generates the synthesized data based on the plurality of pieces of the complementary data and the vehicle data (Paras. 0113-0114; As shown in Fig. 2, imaging device 11D is positioned at the back/rear of the vehicle and imaging devices 11B and 11C are positioned on the sides of the vehicle; As shown in Fig. 16, another vehicle 2 is in back of and to the side of host vehicle 1; Abstract; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13, there is a visible side surface image, and image complementing is performed; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0043: the image complementing unit stores the complemented vehicle periphery image; Paras. 0061-0066: image complementing process; Para. 0065: “the image complementing unit 26 generates a three dimensional image of the retrieved vehicles”; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image as the synthesized data).
Regarding claim 8, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), further comprising: an image generator that generates a predetermined image including the vehicle and the external area of the vehicle, based on the synthesized data generated by the data synthesizer (Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data; and an image reconstruction part 28 for reconstructing the image around the own vehicle after the complement to an image viewed from a viewpoint indicated by virtual viewpoint information on the basis of the virtual viewpoint information input from a virtual viewpoint setting part 27”; Para. 0001: “The present invention relates to a technique for generating a vehicle periphery image having a three dimensional structure viewed from an arbitrary viewpoint”; Para. 0024: visible portions of other vehicles included in the vehicle periphery image are detected; Para. 0043: the image complementing unit stores the complemented vehicle periphery image and inputs the complemented vehicle-periphery image to the image reconstructing unit in which the image is displayed in the set viewpoint; Paras. 0050-0054: reconstructing process; Paras. 0061-0066: image complementing process; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image viewed from a set viewpoint as the predetermined image),
wherein the image generator generates, as the predetermined image, at least one of an image viewed from a side, an image viewed from above, or an image viewed obliquely, with respect to the vehicle (Para. 0040: visible side image is input into the image complementing unit; As shown in Para. 0092: reconstructed image is displayed and the blind spot is complemented by the side image; Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data; and an image reconstruction part 28 for reconstructing the image around the own vehicle after the complement to an image viewed from a viewpoint indicated by virtual viewpoint information on the basis of the virtual viewpoint information input from a virtual viewpoint setting part 27”; Para. 0001: “The present invention relates to a technique for generating a vehicle periphery image having a three dimensional structure viewed from an arbitrary viewpoint”; Para. 0024: visible portions of other vehicles included in the vehicle periphery image are detected; Para. 0043: the image complementing unit stores the complemented vehicle periphery image and inputs the complemented vehicle-periphery image to the image reconstructing unit in which the image is displayed in the set viewpoint; Paras. 0050-0054: reconstructing process; Note: the Examiner selects the image viewed from the side limitation).
Regarding claim 9, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi further teaches, The information processing device according to claim 1 (see claim 1 above), wherein the vehicle data includes at least one of data of the vehicle in a state with a door closed or data of the vehicle in a state with the door open (Para. 0059: “That is, if the other vehicle is in a traveling state, it can be determined that the door on the opposite side is closed, and thus it is determined that the estimation is possible… In other words, when the other vehicle is stopped, the door on the opposite side may be open, and thus it is determined that the estimation is impossible”; Para. 0090; Para. 0105).
Regarding claim 10, Hiroyoshi teaches, An information processing device for generating driving assistance information for a vehicle, the information processing device comprising (Fig. 1: image generation device 100; Abstract: “An image generation device for vehicles comprises: a blind spot information estimating part 25 for estimating information including the shape and the color of another vehicle in blind spot parts on the basis of image information on visible image parts of the another vehicle included in an image around an own vehicle having a three dimensional structure; Para. 0021: “Referring back to FIG. 1, the ROM12 stores a dedicated program for realizing a vehicle periphery image generation function, which will be described later, and various kinds of information necessary for executing the program. In response to a request from the CPU10, the ROM12 reads and inputs the stored information to the CPU10”; Paras. 0034-0035: another vehicle is detected and traveling state is determined):
a detection data obtainer that obtains detection data, the detection data being three-dimensional data obtained by detecting an object located outside the vehicle (Para. 0005: “a vehicle periphery image having a three dimensional structure is generated based on a plurality of images obtained by photographing the periphery of a vehicle by a plurality of cameras mounted on the vehicle”; Para. 0010; Para. 0011: “Here, the vehicular image generating apparatus 100 of the present embodiment generates a three dimensionally structured image of the vehicular periphery by using the image obtained by photographing the region of the vehicular periphery by the imaging devices 11A to 11D… In order to generate a vehicle periphery image having a three dimensional structure, it is necessary to detect three dimensional coordinate information (X, Y, Z) of a subject (object) present in an imaging region in addition to an image captured by a camera”);
and a synthesizer that extracts object data from a plurality of pieces of three-dimensional data of objects stored in advance, the object data being three-dimensional data of the object corresponding to the detection data (Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data”; Para. 0033: “The vehicle-periphery image generating unit 23 generates a vehicle-periphery image having a three dimensional structure… Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13 in association with the photographing time information of the photographed image used for the generation, and inputs the vehicle-periphery image to the vehicle-image detecting unit 24”; Para. 0034: there is template matching in which the vehicle detection unit 24 cuts out an image portion of the detected other vehicle included in the vehicle periphery image data and inputs it to the blind spot information estimation unit 25; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Paras. 0061-0066: image complementing process), and synthesizes the detection data, the object data, and three-dimensional data of the vehicle to generate synthesized data that is three-dimensional data (Abstract; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13 and there is a visible side surface image; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0043: the image complementing unit stores the complemented vehicle periphery image; Paras. 0061-0066: image complementing process; Para. 0065: “the image complementing unit 26 generates a three dimensional image of the retrieved vehicles”; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image as the synthesized data)).
Hiroyoshi discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for obtaining detection data, extracting object data corresponding to the detection data, and synthesizing the detection data, object data, and vehicle data to generate synthesized data since modifications, improvements, equivalents, and the like can be made to the present invention (Hiroyoshi, Para. 0115) and in order to more accurately generate an image of another vehicle (Hiroyoshi, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 10.
Regarding claim 11, Hiroyoshi teaches, An information processing method for generating driving assistance information for a vehicle, the information processing method comprising (Para. 0005: a vehicle periphery image having a three dimensional structure is generated, information of the blind spot is estimated, and the image of the blind spot of the other vehicle is complemented; Para. 0008: “image generation method for a vehicle”; Paras. 0034-0035: another vehicle is detected and traveling state is determined):
obtaining detection data, the detection data being three-dimensional data obtained by detecting an object located outside the vehicle (Para. 0005: “a vehicle periphery image having a three dimensional structure is generated based on a plurality of images obtained by photographing the periphery of a vehicle by a plurality of cameras mounted on the vehicle”; Para. 0010; Para. 0011: “Here, the vehicular image generating apparatus 100 of the present embodiment generates a three dimensionally structured image of the vehicular periphery by using the image obtained by photographing the region of the vehicular periphery by the imaging devices 11A to 11D… In order to generate a vehicle periphery image having a three dimensional structure, it is necessary to detect three dimensional coordinate information (X, Y, Z) of a subject (object) present in an imaging region in addition to an image captured by a camera”);
extracting object data from a plurality of pieces of three-dimensional data of objects stored in advance, the object data being three-dimensional data of the object corresponding to the detection data, and complements the detection data with the object data to generate complementary data that is three-dimensional data (Abstract: “an image complementing part 26 for generating complementary image data for complementing the blind spot parts from the visible image part data on the basis of the blind spot information estimated by the blind spot information estimating part and complementing the image of the blind spot parts of the another vehicle by using the generated complementary image data”; Para. 0033: “The vehicle-periphery image generating unit 23 generates a vehicle-periphery image having a three dimensional structure… Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13 in association with the photographing time information of the photographed image used for the generation, and inputs the vehicle-periphery image to the vehicle-image detecting unit 24”; Para. 0034: there is template matching in which the vehicle detection unit 24 cuts out an image portion of the detected other vehicle included in the vehicle periphery image data and inputs it to the blind spot information estimation unit 25; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Paras. 0061-0066: image complementing process);
and synthesizing the complementary data and vehicle data, the vehicle data being three-dimensional data of the vehicle, to generate synthesized data that is three-dimensional data including the vehicle and an external area of the vehicle (Abstract; Para. 0033: “Specifically, a three dimensional CG model corresponding to the own vehicle 1 is prepared in advance, and the vehicle periphery image data is generated by synthesizing the projection images corresponding to the respective photographing ranges with reference to the own vehicle model. The vehicle-periphery image generating unit 23 stores the generated vehicle-periphery image in the RAM 13”; Para. 0040: the vehicle-surroundings image (i.e., vehicle-periphery image) is stored in RAM 13 and there is a visible side surface image; Para. 0042: “When the image complementing unit 26 determines that the blind spot information estimation unit 25 cannot estimate a blind spot, the image complementing unit 26 searches the car image database 300 stored in the HDD16 for a three dimensional CG model of the same car type as that of other cars. When the three dimensional CG model of the same vehicle type is retrieved by this search, the data is used to generate complementary image data. The image complementing unit 26 complements the image of the blind spot of the other vehicle using the generated complementary image data”; Para. 0043: the image complementing unit stores the complemented vehicle periphery image; Paras. 0061-0066: image complementing process; Para. 0065: “the image complementing unit 26 generates a three dimensional image of the retrieved vehicles”; Note: the Examiner interprets the output complemented vehicle periphery image/generated three dimensional image as the synthesized data).
Hiroyoshi discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for obtaining detection data, complementing the detection data with extracted object data to generate complementary data, and synthesizing the complementary data and vehicle data to generate synthesized data since modifications, improvements, equivalents, and the like can be made to the present invention (Hiroyoshi, Para. 0115) and in order to more accurately generate an image of another vehicle (Hiroyoshi, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 11.
Regarding claim 12, Hiroyoshi teaches the limitations as explained above in claim 11.
Hiroyoshi further teaches, A non-transitory computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the information processing method according to claim 11 (Fig. 1: image generation device 100, ROM 12, RAM 13, CPU 10; Para. 0021: “Referring back to FIG. 1, the ROM 12 stores a dedicated program for realizing a vehicle periphery image generation function, which will be described later, and various kinds of information necessary for executing the program. In response to a request from the CPU 10, the ROM 12 reads and inputs the stored information to the CPU 10”; Paras. 0022-0024; see claim 11 above).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Hiroyoshi (JP 2013025528 A, see provided machine translation) in view of Omori et al. (US 2019/0344828 A1; hereinafter “Omori”).
Regarding claim 6, Hiroyoshi teaches the limitations as explained above in claim 1.
Hiroyoshi does not expressly disclose the following limitation: wherein the information processing device generates the driving assistance information when the vehicle is parked.
However, Omori teaches, wherein the information processing device generates the driving assistance information when the vehicle is parked (Para. 0037:” As illustrated in FIG. 1, the parking assist apparatus includes a parking assist ECU 10. The parking assist ECU 10 includes a microcomputer including a central processing unit (CPU) 10a, a random-access memory (RAM) 10b, a read-only memory (ROM) 10c, an interface (I/F) 10d, and other component”; Para. 0050: “Therefore, the cameras 83 are configured to photograph a peripheral state (including the position, shape, and the like of the separation lines, objects, parkable regions, and the like) of the vehicle to be confirmed when the vehicle is parked, and output image data to the parking assist ECU 10”; Paras. 0059-0061; Para. 0081: “When it is determined that the above-mentioned parking assist request has been issued, the parking assist ECU 10 sets the position of the own vehicle at the time when the own vehicle is assumed to be parked in the candidate region (e.g., a center position in plan view of the left front wheel and the right front wheel of the own vehicle) as the target position. The parking assist ECU 10 determines/sets, as a target path, a path along which the position of the own vehicle is to be moved from the current own vehicle position (current position) to the target position. The parking assist ECU 10 executes parking assist control such that the vehicle moves along the target path”).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine the information processing device generating driving assistance information when the vehicle is parked as taught by Omori with the method of Hiroyoshi in order to improve visibility exhibited when a driver confirms a final parking state of a vehicle (Omori, Para. 0007). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 6.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
JP 2012051391A (see provided machine translation)
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniella M. DiGuglielmo whose telephone number is (571)272-0183. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniella M. DiGuglielmo/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666